Sep 10 13:18:10.003691 ip-10-0-142-118 systemd[1]: Starting Kubernetes Kubelet... Sep 10 13:18:10.347871 ip-10-0-142-118 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 13:18:10.347871 ip-10-0-142-118 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 10 13:18:10.347871 ip-10-0-142-118 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 13:18:10.347871 ip-10-0-142-118 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 10 13:18:10.347871 ip-10-0-142-118 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 13:18:10.350264 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.350144 2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 10 13:18:10.352952 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.352936 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 13:18:10.352952 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.352953 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 13:18:10.353014 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.352957 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 13:18:10.353014 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.352962 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 13:18:10.353014 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.352966 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 13:18:10.353014 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.352969 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 13:18:10.353014 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.352973 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 13:18:10.353014 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.352976 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 13:18:10.353014 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.352979 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 13:18:10.353014 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.352982 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 13:18:10.353014 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.352992 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 13:18:10.353014 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.352994 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 13:18:10.353014 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.352998 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 13:18:10.353014 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353002 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 13:18:10.353014 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353005 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 13:18:10.353014 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353008 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 13:18:10.353014 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353011 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 13:18:10.353014 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353013 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 13:18:10.353014 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353016 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 13:18:10.353014 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353019 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 13:18:10.353014 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353022 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 13:18:10.353511 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353025 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 13:18:10.353511 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353028 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 13:18:10.353511 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353030 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 13:18:10.353511 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353033 2560 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 13:18:10.353511 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353036 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 13:18:10.353511 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353038 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 13:18:10.353511 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353043 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 13:18:10.353511 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353046 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 13:18:10.353511 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353049 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 13:18:10.353511 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353052 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 13:18:10.353511 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353055 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 13:18:10.353511 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353057 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 13:18:10.353511 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353060 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 13:18:10.353511 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353062 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 13:18:10.353511 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353065 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 13:18:10.353511 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353067 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 13:18:10.353511 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353070 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 13:18:10.353511 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353072 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 13:18:10.353511 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353075 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 13:18:10.353511 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353078 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 13:18:10.353994 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353080 2560 feature_gate.go:328] unrecognized feature gate: Example Sep 10 13:18:10.353994 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353083 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 13:18:10.353994 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353085 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 13:18:10.353994 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353088 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 13:18:10.353994 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353090 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 13:18:10.353994 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353093 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 13:18:10.353994 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353096 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 13:18:10.353994 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353099 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 13:18:10.353994 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353101 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 13:18:10.353994 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353104 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 13:18:10.353994 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353106 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 13:18:10.353994 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353109 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 13:18:10.353994 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353111 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 13:18:10.353994 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353114 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 13:18:10.353994 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353117 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 13:18:10.353994 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353120 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 13:18:10.353994 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353122 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 13:18:10.353994 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353125 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 13:18:10.353994 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353127 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 13:18:10.353994 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353130 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 13:18:10.354587 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353132 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 13:18:10.354587 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353135 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 13:18:10.354587 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353137 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 13:18:10.354587 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353140 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 13:18:10.354587 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353142 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 13:18:10.354587 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353144 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 13:18:10.354587 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353147 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 13:18:10.354587 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353149 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 13:18:10.354587 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353152 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 13:18:10.354587 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353154 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 13:18:10.354587 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353159 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 13:18:10.354587 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353162 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 13:18:10.354587 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353164 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 13:18:10.354587 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353167 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 13:18:10.354587 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353169 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 13:18:10.354587 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353172 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 13:18:10.354587 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353174 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 13:18:10.354587 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353177 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 13:18:10.354587 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353179 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 13:18:10.355087 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353181 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 13:18:10.355087 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353184 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 13:18:10.355087 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353186 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 13:18:10.355087 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353189 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 13:18:10.355087 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353191 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 13:18:10.355087 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353194 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 13:18:10.355087 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353197 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 13:18:10.355087 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353200 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 13:18:10.355087 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353202 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 13:18:10.355087 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353205 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 13:18:10.355087 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353207 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 13:18:10.355087 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353209 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 13:18:10.355087 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353212 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 13:18:10.355087 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353214 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 13:18:10.355087 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353217 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 13:18:10.355087 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353219 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 13:18:10.355087 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353222 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 13:18:10.355087 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353224 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 13:18:10.355087 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353226 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 13:18:10.355087 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353931 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 13:18:10.355583 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353938 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 13:18:10.355583 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353941 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 13:18:10.355583 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353944 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 13:18:10.355583 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353948 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 13:18:10.355583 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353952 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 13:18:10.355583 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353955 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 13:18:10.355583 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353958 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 13:18:10.355583 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353960 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 13:18:10.355583 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353963 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 13:18:10.355583 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353965 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 13:18:10.355583 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353968 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 13:18:10.355583 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353971 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 13:18:10.355583 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353973 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 13:18:10.355583 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353976 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 13:18:10.355583 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353978 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 13:18:10.355583 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353981 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 13:18:10.355583 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353984 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 13:18:10.355583 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353986 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 13:18:10.355583 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353991 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 13:18:10.355583 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353994 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 13:18:10.356085 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.353997 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 13:18:10.356085 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354000 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 13:18:10.356085 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354003 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 13:18:10.356085 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354006 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 13:18:10.356085 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354009 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 13:18:10.356085 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354011 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 13:18:10.356085 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354014 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 13:18:10.356085 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354017 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 13:18:10.356085 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354019 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 13:18:10.356085 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354021 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 13:18:10.356085 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354024 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 13:18:10.356085 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354026 2560 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 13:18:10.356085 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354029 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 13:18:10.356085 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354032 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 13:18:10.356085 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354034 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 13:18:10.356085 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354036 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 13:18:10.356085 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354039 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 13:18:10.356085 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354042 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 13:18:10.356085 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354045 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 13:18:10.356085 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354047 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 13:18:10.356576 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354049 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 13:18:10.356576 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354052 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 13:18:10.356576 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354055 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 13:18:10.356576 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354057 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 13:18:10.356576 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354060 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 13:18:10.356576 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354062 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 13:18:10.356576 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354065 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 13:18:10.356576 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354067 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 13:18:10.356576 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354070 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 13:18:10.356576 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354073 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 13:18:10.356576 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354076 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 13:18:10.356576 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354079 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 13:18:10.356576 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354081 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 13:18:10.356576 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354084 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 13:18:10.356576 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354087 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 13:18:10.356576 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354089 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 13:18:10.356576 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354092 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 13:18:10.356576 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354094 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 13:18:10.356576 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354097 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 13:18:10.356576 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354099 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 13:18:10.357066 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354102 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 13:18:10.357066 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354104 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 13:18:10.357066 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354107 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 13:18:10.357066 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354111 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 13:18:10.357066 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354114 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 13:18:10.357066 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354118 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 13:18:10.357066 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354120 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 13:18:10.357066 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354123 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 13:18:10.357066 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354126 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 13:18:10.357066 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354128 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 13:18:10.357066 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354131 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 13:18:10.357066 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354134 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 13:18:10.357066 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354137 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 13:18:10.357066 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354139 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 13:18:10.357066 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354142 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 13:18:10.357066 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354145 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 13:18:10.357066 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354147 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 13:18:10.357066 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354151 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 13:18:10.357066 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354153 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 13:18:10.357542 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354156 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 13:18:10.357542 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354158 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 13:18:10.357542 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354161 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 13:18:10.357542 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354164 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 13:18:10.357542 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354167 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 13:18:10.357542 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354169 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 13:18:10.357542 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354172 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 13:18:10.357542 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354174 2560 feature_gate.go:328] unrecognized feature gate: Example Sep 10 13:18:10.357542 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354177 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 13:18:10.357542 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354181 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 13:18:10.357542 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354184 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 13:18:10.357542 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354187 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 13:18:10.357542 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354190 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 13:18:10.357542 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354192 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 13:18:10.357542 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354195 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 13:18:10.357542 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354197 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 13:18:10.357542 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354200 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 13:18:10.357542 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354202 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 13:18:10.357542 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354205 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 13:18:10.358005 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354607 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 13:18:10.358005 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354613 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 13:18:10.358005 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354616 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 13:18:10.358005 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354618 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 13:18:10.358005 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354622 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 13:18:10.358005 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354625 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 13:18:10.358005 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354628 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 13:18:10.358005 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354630 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 13:18:10.358005 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354633 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 13:18:10.358005 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354635 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 13:18:10.358005 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354638 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 13:18:10.358005 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354641 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 13:18:10.358005 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354643 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 13:18:10.358005 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354646 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 13:18:10.358005 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354649 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 13:18:10.358005 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354651 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 13:18:10.358005 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354654 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 13:18:10.358005 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354657 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 13:18:10.358005 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354660 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 13:18:10.358005 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354663 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 13:18:10.358554 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354665 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 13:18:10.358554 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354669 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 13:18:10.358554 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354671 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 13:18:10.358554 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354674 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 13:18:10.358554 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354676 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 13:18:10.358554 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354679 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 13:18:10.358554 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354681 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 13:18:10.358554 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354684 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 13:18:10.358554 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354686 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 13:18:10.358554 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354689 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 13:18:10.358554 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354691 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 13:18:10.358554 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354694 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 13:18:10.358554 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354697 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 13:18:10.358554 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354701 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 13:18:10.358554 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354704 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 13:18:10.358554 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354707 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 13:18:10.358554 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354710 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 13:18:10.358554 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354714 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 13:18:10.358554 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354717 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 13:18:10.359024 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354720 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 13:18:10.359024 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354722 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 13:18:10.359024 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354727 2560 feature_gate.go:328] unrecognized feature gate: Example Sep 10 13:18:10.359024 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354731 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 13:18:10.359024 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354733 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 13:18:10.359024 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354736 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 13:18:10.359024 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354738 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 13:18:10.359024 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354741 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 13:18:10.359024 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354744 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 13:18:10.359024 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354747 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 13:18:10.359024 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354750 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 13:18:10.359024 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354752 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 13:18:10.359024 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354755 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 13:18:10.359024 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354757 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 13:18:10.359024 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354760 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 13:18:10.359024 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354763 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 13:18:10.359024 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354767 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 13:18:10.359024 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354769 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 13:18:10.359024 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354772 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 13:18:10.359024 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354775 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 13:18:10.359529 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354778 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 13:18:10.359529 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354780 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 13:18:10.359529 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354783 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 13:18:10.359529 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354786 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 13:18:10.359529 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354788 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 13:18:10.359529 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354791 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 13:18:10.359529 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354793 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 13:18:10.359529 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354796 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 13:18:10.359529 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354799 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 13:18:10.359529 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354801 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 13:18:10.359529 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354804 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 13:18:10.359529 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354813 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 13:18:10.359529 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354816 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 13:18:10.359529 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354819 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 13:18:10.359529 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354824 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 13:18:10.359529 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354829 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 13:18:10.359529 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354832 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 13:18:10.359529 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354835 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 13:18:10.359529 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354838 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 13:18:10.360016 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354841 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 13:18:10.360016 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354844 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 13:18:10.360016 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354847 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 13:18:10.360016 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354849 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 13:18:10.360016 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354852 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 13:18:10.360016 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354855 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 13:18:10.360016 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354857 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 13:18:10.360016 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354860 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 13:18:10.360016 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354863 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 13:18:10.360016 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354865 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 13:18:10.360016 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354868 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 13:18:10.360016 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354871 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 13:18:10.360016 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354873 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 13:18:10.360016 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354876 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 13:18:10.360016 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354878 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 13:18:10.360016 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354880 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 13:18:10.360016 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354883 2560 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 13:18:10.360016 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354886 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 13:18:10.360016 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354888 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 13:18:10.360016 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354890 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 13:18:10.360509 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.354893 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 13:18:10.360509 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.354968 2560 flags.go:64] FLAG: --address="0.0.0.0" Sep 10 13:18:10.360509 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355002 2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 10 13:18:10.360509 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355010 2560 flags.go:64] FLAG: --anonymous-auth="true" Sep 10 13:18:10.360509 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355015 2560 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 10 13:18:10.360509 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355020 2560 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 10 13:18:10.360509 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355024 2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 10 13:18:10.360509 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355029 2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 10 13:18:10.360509 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355034 2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 10 13:18:10.360509 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355037 2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 10 13:18:10.360509 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355040 2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 10 13:18:10.360509 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355044 2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 10 13:18:10.360509 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355047 2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 10 13:18:10.360509 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355051 2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 10 13:18:10.360509 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355054 2560 flags.go:64] FLAG: --cgroup-root="" Sep 10 13:18:10.360509 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355056 2560 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 10 13:18:10.360509 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355060 2560 flags.go:64] FLAG: --client-ca-file="" Sep 10 13:18:10.360509 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355063 2560 flags.go:64] FLAG: --cloud-provider="external" Sep 10 13:18:10.360509 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355066 2560 flags.go:64] FLAG: --cluster-dns="[]" Sep 10 13:18:10.360509 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355069 2560 flags.go:64] FLAG: --cluster-domain="" Sep 10 13:18:10.360509 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355072 2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 10 13:18:10.360509 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355075 2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 10 13:18:10.360509 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355079 2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 10 13:18:10.360509 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355082 2560 flags.go:64] FLAG: --container-log-max-files="5" Sep 10 13:18:10.361141 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355086 2560 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 10 13:18:10.361141 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355089 2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 10 13:18:10.361141 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355092 2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 10 13:18:10.361141 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355096 2560 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 10 13:18:10.361141 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355099 2560 flags.go:64] FLAG: --contention-profiling="false" Sep 10 13:18:10.361141 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355102 2560 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 10 13:18:10.361141 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355107 2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 10 13:18:10.361141 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355110 2560 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 10 13:18:10.361141 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355113 2560 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 10 13:18:10.361141 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355117 2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 10 13:18:10.361141 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355121 2560 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 10 13:18:10.361141 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355123 2560 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 10 13:18:10.361141 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355126 2560 flags.go:64] FLAG: --enable-load-reader="false" Sep 10 13:18:10.361141 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355129 2560 flags.go:64] FLAG: --enable-server="true" Sep 10 13:18:10.361141 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355133 2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 10 13:18:10.361141 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355139 2560 flags.go:64] FLAG: --event-burst="100" Sep 10 13:18:10.361141 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355142 2560 flags.go:64] FLAG: --event-qps="50" Sep 10 13:18:10.361141 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355145 2560 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 10 13:18:10.361141 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355148 2560 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 10 13:18:10.361141 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355151 2560 flags.go:64] FLAG: --eviction-hard="" Sep 10 13:18:10.361141 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355155 2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 10 13:18:10.361141 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355157 2560 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 10 13:18:10.361141 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355160 2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 10 13:18:10.361141 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355163 2560 flags.go:64] FLAG: --eviction-soft="" Sep 10 13:18:10.361141 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355166 2560 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 10 13:18:10.361782 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355169 2560 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 10 13:18:10.361782 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355172 2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 10 13:18:10.361782 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355175 2560 flags.go:64] FLAG: --experimental-mounter-path="" Sep 10 13:18:10.361782 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355178 2560 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 10 13:18:10.361782 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355180 2560 flags.go:64] FLAG: --fail-swap-on="true" Sep 10 13:18:10.361782 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355183 2560 flags.go:64] FLAG: --feature-gates="" Sep 10 13:18:10.361782 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355189 2560 flags.go:64] FLAG: --file-check-frequency="20s" Sep 10 13:18:10.361782 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355193 2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 10 13:18:10.361782 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355196 2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 10 13:18:10.361782 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355199 2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 10 13:18:10.361782 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355202 2560 flags.go:64] FLAG: --healthz-port="10248" Sep 10 13:18:10.361782 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355204 2560 flags.go:64] FLAG: --help="false" Sep 10 13:18:10.361782 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355207 2560 flags.go:64] FLAG: --hostname-override="ip-10-0-142-118.ec2.internal" Sep 10 13:18:10.361782 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355212 2560 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 10 13:18:10.361782 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355215 2560 flags.go:64] FLAG: --http-check-frequency="20s" Sep 10 13:18:10.361782 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355218 2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 10 13:18:10.361782 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355222 2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 10 13:18:10.361782 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355225 2560 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 10 13:18:10.361782 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355228 2560 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 10 13:18:10.361782 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355231 2560 flags.go:64] FLAG: --image-service-endpoint="" Sep 10 13:18:10.361782 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355234 2560 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 10 13:18:10.361782 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355236 2560 flags.go:64] FLAG: --kube-api-burst="100" Sep 10 13:18:10.361782 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355241 2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 10 13:18:10.362378 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355244 2560 flags.go:64] FLAG: --kube-api-qps="50" Sep 10 13:18:10.362378 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355247 2560 flags.go:64] FLAG: --kube-reserved="" Sep 10 13:18:10.362378 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355250 2560 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 10 13:18:10.362378 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355254 2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 10 13:18:10.362378 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355257 2560 flags.go:64] FLAG: --kubelet-cgroups="" Sep 10 13:18:10.362378 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355259 2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 10 13:18:10.362378 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355262 2560 flags.go:64] FLAG: --lock-file="" Sep 10 13:18:10.362378 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355265 2560 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 10 13:18:10.362378 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355268 2560 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 10 13:18:10.362378 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355271 2560 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 10 13:18:10.362378 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355276 2560 flags.go:64] FLAG: --log-json-split-stream="false" Sep 10 13:18:10.362378 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355279 2560 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 10 13:18:10.362378 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355282 2560 flags.go:64] FLAG: --log-text-split-stream="false" Sep 10 13:18:10.362378 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355285 2560 flags.go:64] FLAG: --logging-format="text" Sep 10 13:18:10.362378 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355288 2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 10 13:18:10.362378 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355291 2560 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 10 13:18:10.362378 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355294 2560 flags.go:64] FLAG: --manifest-url="" Sep 10 13:18:10.362378 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355296 2560 flags.go:64] FLAG: --manifest-url-header="" Sep 10 13:18:10.362378 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355301 2560 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 10 13:18:10.362378 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355304 2560 flags.go:64] FLAG: --max-open-files="1000000" Sep 10 13:18:10.362378 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355308 2560 flags.go:64] FLAG: --max-pods="110" Sep 10 13:18:10.362378 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355311 2560 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 10 13:18:10.362378 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355315 2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 10 13:18:10.362378 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355318 2560 flags.go:64] FLAG: --memory-manager-policy="None" Sep 10 13:18:10.362378 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355321 2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 10 13:18:10.363006 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355324 2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 10 13:18:10.363006 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355326 2560 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 10 13:18:10.363006 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355329 2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 10 13:18:10.363006 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355337 2560 flags.go:64] FLAG: --node-status-max-images="50" Sep 10 13:18:10.363006 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355340 2560 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 10 13:18:10.363006 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355343 2560 flags.go:64] FLAG: --oom-score-adj="-999" Sep 10 13:18:10.363006 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355347 2560 flags.go:64] FLAG: --pod-cidr="" Sep 10 13:18:10.363006 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355350 2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 10 13:18:10.363006 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355355 2560 flags.go:64] FLAG: --pod-manifest-path="" Sep 10 13:18:10.363006 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355358 2560 flags.go:64] FLAG: --pod-max-pids="-1" Sep 10 13:18:10.363006 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355362 2560 flags.go:64] FLAG: --pods-per-core="0" Sep 10 13:18:10.363006 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355364 2560 flags.go:64] FLAG: --port="10250" Sep 10 13:18:10.363006 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355367 2560 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 10 13:18:10.363006 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355370 2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-05d9ebf737c7544a7" Sep 10 13:18:10.363006 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355374 2560 flags.go:64] FLAG: --qos-reserved="" Sep 10 13:18:10.363006 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355377 2560 flags.go:64] FLAG: --read-only-port="10255" Sep 10 13:18:10.363006 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355380 2560 flags.go:64] FLAG: --register-node="true" Sep 10 13:18:10.363006 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355382 2560 flags.go:64] FLAG: --register-with-taints="" Sep 10 13:18:10.363006 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355386 2560 flags.go:64] FLAG: --registry-burst="10" Sep 10 13:18:10.363006 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355388 2560 flags.go:64] FLAG: --registry-qps="5" Sep 10 13:18:10.363006 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355392 2560 flags.go:64] FLAG: --reserved-cpus="" Sep 10 13:18:10.363006 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355395 2560 flags.go:64] FLAG: --reserved-memory="" Sep 10 13:18:10.363006 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355410 2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 10 13:18:10.363006 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355414 2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 10 13:18:10.363006 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355417 2560 flags.go:64] FLAG: --rotate-certificates="false" Sep 10 13:18:10.363626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355420 2560 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 10 13:18:10.363626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355423 2560 flags.go:64] FLAG: --runonce="false" Sep 10 13:18:10.363626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355426 2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 10 13:18:10.363626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355429 2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 10 13:18:10.363626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355434 2560 flags.go:64] FLAG: --seccomp-default="false" Sep 10 13:18:10.363626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355437 2560 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 10 13:18:10.363626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355440 2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 10 13:18:10.363626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355443 2560 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 10 13:18:10.363626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355446 2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 10 13:18:10.363626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355449 2560 flags.go:64] FLAG: --storage-driver-password="root" Sep 10 13:18:10.363626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355452 2560 flags.go:64] FLAG: --storage-driver-secure="false" Sep 10 13:18:10.363626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355455 2560 flags.go:64] FLAG: --storage-driver-table="stats" Sep 10 13:18:10.363626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355457 2560 flags.go:64] FLAG: --storage-driver-user="root" Sep 10 13:18:10.363626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355460 2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 10 13:18:10.363626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355464 2560 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 10 13:18:10.363626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355468 2560 flags.go:64] FLAG: --system-cgroups="" Sep 10 13:18:10.363626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355472 2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 10 13:18:10.363626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355477 2560 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 10 13:18:10.363626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355480 2560 flags.go:64] FLAG: --tls-cert-file="" Sep 10 13:18:10.363626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355482 2560 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 10 13:18:10.363626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355486 2560 flags.go:64] FLAG: --tls-min-version="" Sep 10 13:18:10.363626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355489 2560 flags.go:64] FLAG: --tls-private-key-file="" Sep 10 13:18:10.363626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355492 2560 flags.go:64] FLAG: --topology-manager-policy="none" Sep 10 13:18:10.363626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355495 2560 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 10 13:18:10.363626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355498 2560 flags.go:64] FLAG: --topology-manager-scope="container" Sep 10 13:18:10.364225 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355501 2560 flags.go:64] FLAG: --v="2" Sep 10 13:18:10.364225 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355505 2560 flags.go:64] FLAG: --version="false" Sep 10 13:18:10.364225 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355508 2560 flags.go:64] FLAG: --vmodule="" Sep 10 13:18:10.364225 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355512 2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 10 13:18:10.364225 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355515 2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 10 13:18:10.364225 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355636 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 13:18:10.364225 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355640 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 13:18:10.364225 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355643 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 13:18:10.364225 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355646 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 13:18:10.364225 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355649 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 13:18:10.364225 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355653 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 13:18:10.364225 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355659 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 13:18:10.364225 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355662 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 13:18:10.364225 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355664 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 13:18:10.364225 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355667 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 13:18:10.364225 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355671 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 13:18:10.364225 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355674 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 13:18:10.364225 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355677 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 13:18:10.364225 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355679 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 13:18:10.364225 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355682 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 13:18:10.364759 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355684 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 13:18:10.364759 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355688 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 13:18:10.364759 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355691 2560 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 13:18:10.364759 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355694 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 13:18:10.364759 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355697 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 13:18:10.364759 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355700 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 13:18:10.364759 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355703 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 13:18:10.364759 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355705 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 13:18:10.364759 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355708 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 13:18:10.364759 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355711 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 13:18:10.364759 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355713 2560 feature_gate.go:328] unrecognized feature gate: Example Sep 10 13:18:10.364759 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355716 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 13:18:10.364759 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355719 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 13:18:10.364759 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355721 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 13:18:10.364759 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355724 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 13:18:10.364759 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355727 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 13:18:10.364759 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355729 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 13:18:10.364759 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355732 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 13:18:10.364759 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355735 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 13:18:10.364759 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355737 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 13:18:10.365251 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355739 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 13:18:10.365251 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355742 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 13:18:10.365251 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355745 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 13:18:10.365251 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355749 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 13:18:10.365251 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355751 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 13:18:10.365251 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355753 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 13:18:10.365251 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355756 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 13:18:10.365251 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355759 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 13:18:10.365251 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355761 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 13:18:10.365251 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355764 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 13:18:10.365251 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355766 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 13:18:10.365251 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355769 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 13:18:10.365251 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355771 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 13:18:10.365251 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355775 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 13:18:10.365251 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355777 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 13:18:10.365251 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355780 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 13:18:10.365251 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355783 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 13:18:10.365251 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355786 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 13:18:10.365251 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355789 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 13:18:10.365251 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355791 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 13:18:10.365817 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355794 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 13:18:10.365817 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355797 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 13:18:10.365817 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355799 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 13:18:10.365817 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355802 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 13:18:10.365817 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355804 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 13:18:10.365817 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355807 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 13:18:10.365817 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355811 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 13:18:10.365817 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355814 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 13:18:10.365817 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355817 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 13:18:10.365817 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355820 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 13:18:10.365817 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355822 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 13:18:10.365817 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355825 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 13:18:10.365817 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355827 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 13:18:10.365817 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355830 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 13:18:10.365817 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355833 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 13:18:10.365817 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355837 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 13:18:10.365817 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355839 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 13:18:10.365817 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355842 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 13:18:10.365817 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355845 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 13:18:10.366289 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355848 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 13:18:10.366289 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355851 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 13:18:10.366289 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355855 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 13:18:10.366289 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355858 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 13:18:10.366289 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355860 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 13:18:10.366289 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355863 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 13:18:10.366289 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355867 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 13:18:10.366289 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355869 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 13:18:10.366289 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355872 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 13:18:10.366289 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355875 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 13:18:10.366289 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355878 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 13:18:10.366289 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355881 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 13:18:10.366289 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355883 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 13:18:10.366289 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355886 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 13:18:10.366289 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355888 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 13:18:10.366289 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355891 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 13:18:10.366289 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355894 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 13:18:10.366289 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355896 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 13:18:10.366289 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355899 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 13:18:10.366289 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355902 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 13:18:10.366798 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355904 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 13:18:10.366798 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355907 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 13:18:10.366798 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355910 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 13:18:10.366798 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355912 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 13:18:10.366798 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.355915 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 13:18:10.366798 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.355920 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 13:18:10.366798 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.362575 2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 10 13:18:10.366798 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.362591 2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 10 13:18:10.366798 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.362610 2560 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 10 13:18:10.366798 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.362619 2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 10 13:18:10.366798 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362676 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 13:18:10.366798 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362681 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 13:18:10.366798 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362685 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 13:18:10.366798 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362687 2560 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 13:18:10.366798 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362690 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 13:18:10.367168 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362693 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 13:18:10.367168 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362696 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 13:18:10.367168 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362700 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 13:18:10.367168 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362702 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 13:18:10.367168 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362705 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 13:18:10.367168 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362708 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 13:18:10.367168 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362712 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 13:18:10.367168 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362717 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 13:18:10.367168 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362720 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 13:18:10.367168 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362723 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 13:18:10.367168 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362726 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 13:18:10.367168 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362729 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 13:18:10.367168 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362732 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 13:18:10.367168 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362735 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 13:18:10.367168 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362737 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 13:18:10.367168 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362740 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 13:18:10.367168 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362743 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 13:18:10.367168 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362745 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 13:18:10.367168 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362748 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 13:18:10.367640 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362751 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 13:18:10.367640 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362753 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 13:18:10.367640 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362756 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 13:18:10.367640 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362759 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 13:18:10.367640 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362761 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 13:18:10.367640 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362764 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 13:18:10.367640 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362767 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 13:18:10.367640 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362770 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 13:18:10.367640 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362773 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 13:18:10.367640 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362776 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 13:18:10.367640 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362778 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 13:18:10.367640 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362781 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 13:18:10.367640 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362783 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 13:18:10.367640 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362786 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 13:18:10.367640 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362789 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 13:18:10.367640 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362791 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 13:18:10.367640 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362794 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 13:18:10.367640 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362796 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 13:18:10.367640 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362799 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 13:18:10.367640 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362802 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 13:18:10.368164 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362804 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 13:18:10.368164 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362808 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 13:18:10.368164 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362811 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 13:18:10.368164 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362814 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 13:18:10.368164 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362818 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 13:18:10.368164 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362822 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 13:18:10.368164 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362824 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 13:18:10.368164 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362827 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 13:18:10.368164 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362830 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 13:18:10.368164 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362833 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 13:18:10.368164 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362835 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 13:18:10.368164 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362838 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 13:18:10.368164 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362840 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 13:18:10.368164 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362843 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 13:18:10.368164 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362846 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 13:18:10.368164 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362848 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 13:18:10.368164 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362851 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 13:18:10.368164 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362854 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 13:18:10.368164 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362857 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 13:18:10.368164 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362860 2560 feature_gate.go:328] unrecognized feature gate: Example Sep 10 13:18:10.368668 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362862 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 13:18:10.368668 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362865 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 13:18:10.368668 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362868 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 13:18:10.368668 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362870 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 13:18:10.368668 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362873 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 13:18:10.368668 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362876 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 13:18:10.368668 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362879 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 13:18:10.368668 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362882 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 13:18:10.368668 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362884 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 13:18:10.368668 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362887 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 13:18:10.368668 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362889 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 13:18:10.368668 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362892 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 13:18:10.368668 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362895 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 13:18:10.368668 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362897 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 13:18:10.368668 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362900 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 13:18:10.368668 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362903 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 13:18:10.368668 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362905 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 13:18:10.368668 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362908 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 13:18:10.368668 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362911 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 13:18:10.368668 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362914 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 13:18:10.369245 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362916 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 13:18:10.369245 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362919 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 13:18:10.369245 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362921 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 13:18:10.369245 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362923 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 13:18:10.369245 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362926 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 13:18:10.369245 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362929 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 13:18:10.369245 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362931 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 13:18:10.369245 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362934 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 13:18:10.369245 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362938 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 13:18:10.369245 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362941 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 13:18:10.369245 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362945 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 13:18:10.369245 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362948 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 13:18:10.369245 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362951 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 13:18:10.369245 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362953 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 13:18:10.369245 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.362956 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 13:18:10.369896 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.362962 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 13:18:10.369896 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363092 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 13:18:10.369896 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363097 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 13:18:10.369896 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363101 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 13:18:10.369896 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363103 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 13:18:10.369896 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363106 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 13:18:10.369896 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363109 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 13:18:10.369896 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363111 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 13:18:10.369896 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363114 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 13:18:10.369896 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363117 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 13:18:10.369896 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363119 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 13:18:10.369896 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363122 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 13:18:10.369896 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363125 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 13:18:10.369896 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363128 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 13:18:10.369896 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363130 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 13:18:10.370281 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363133 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 13:18:10.370281 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363135 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 13:18:10.370281 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363138 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 13:18:10.370281 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363140 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 13:18:10.370281 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363143 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 13:18:10.370281 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363145 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 13:18:10.370281 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363148 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 13:18:10.370281 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363150 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 13:18:10.370281 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363153 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 13:18:10.370281 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363155 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 13:18:10.370281 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363157 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 13:18:10.370281 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363160 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 13:18:10.370281 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363164 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 13:18:10.370281 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363166 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 13:18:10.370281 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363169 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 13:18:10.370281 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363172 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 13:18:10.370281 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363175 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 13:18:10.370281 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363179 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 13:18:10.370281 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363183 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 13:18:10.370816 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363185 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 13:18:10.370816 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363188 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 13:18:10.370816 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363191 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 13:18:10.370816 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363193 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 13:18:10.370816 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363196 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 13:18:10.370816 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363198 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 13:18:10.370816 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363201 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 13:18:10.370816 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363204 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 13:18:10.370816 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363207 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 13:18:10.370816 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363211 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 13:18:10.370816 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363214 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 13:18:10.370816 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363217 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 13:18:10.370816 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363220 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 13:18:10.370816 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363223 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 13:18:10.370816 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363226 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 13:18:10.370816 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363228 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 13:18:10.370816 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363231 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 13:18:10.370816 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363233 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 13:18:10.370816 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363236 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 13:18:10.371297 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363238 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 13:18:10.371297 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363241 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 13:18:10.371297 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363243 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 13:18:10.371297 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363245 2560 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 13:18:10.371297 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363248 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 13:18:10.371297 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363251 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 13:18:10.371297 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363253 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 13:18:10.371297 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363256 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 13:18:10.371297 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363258 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 13:18:10.371297 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363261 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 13:18:10.371297 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363264 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 13:18:10.371297 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363266 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 13:18:10.371297 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363268 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 13:18:10.371297 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363271 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 13:18:10.371297 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363273 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 13:18:10.371297 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363276 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 13:18:10.371297 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363278 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 13:18:10.371297 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363281 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 13:18:10.371297 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363283 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 13:18:10.371297 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363286 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 13:18:10.371794 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363288 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 13:18:10.371794 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363290 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 13:18:10.371794 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363293 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 13:18:10.371794 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363295 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 13:18:10.371794 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363298 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 13:18:10.371794 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363301 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 13:18:10.371794 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363303 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 13:18:10.371794 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363306 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 13:18:10.371794 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363308 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 13:18:10.371794 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363310 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 13:18:10.371794 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363313 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 13:18:10.371794 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363315 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 13:18:10.371794 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363318 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 13:18:10.371794 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363320 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 13:18:10.371794 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363323 2560 feature_gate.go:328] unrecognized feature gate: Example Sep 10 13:18:10.371794 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363325 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 13:18:10.371794 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363327 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 13:18:10.371794 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363330 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 13:18:10.371794 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363333 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 13:18:10.371794 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363335 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 13:18:10.371794 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363338 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 13:18:10.372302 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363340 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 13:18:10.372302 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363343 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 13:18:10.372302 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363345 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 13:18:10.372302 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363349 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 13:18:10.372302 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363352 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 13:18:10.372302 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:10.363355 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 13:18:10.372302 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.363360 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 13:18:10.372302 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.363482 2560 server.go:962] "Client rotation is on, will bootstrap in background" Sep 10 13:18:10.372302 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.365264 2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 10 13:18:10.372302 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.366344 2560 server.go:1019] "Starting client certificate rotation" Sep 10 13:18:10.372302 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.366449 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 13:18:10.372302 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.366484 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 13:18:10.372302 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.370915 2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 13:18:10.372644 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.372045 2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 13:18:10.380703 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.380683 2560 log.go:25] "Validated CRI v1 runtime API" Sep 10 13:18:10.383367 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.383348 2560 log.go:25] "Validated CRI v1 image API" Sep 10 13:18:10.384165 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.384150 2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 10 13:18:10.386905 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.386876 2560 fs.go:135] Filesystem UUIDs: map[482f733d-6080-40d4-bdf3-1bae3e6eb3c6:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 df7fdb90-1b2f-4455-86b2-a8054aa3f2a3:/dev/nvme0n1p4] Sep 10 13:18:10.386977 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.386904 2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 10 13:18:10.389356 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.389331 2560 certificate_manager.go:596] "Failed while requesting a signed certificate from the control plane" err="cannot create certificate signing request: certificatesigningrequests.certificates.k8s.io is forbidden: User \"system:serviceaccount:openshift-machine-config-operator:node-bootstrapper\" cannot create resource \"certificatesigningrequests\" in API group \"certificates.k8s.io\" at the cluster scope" logger="kubernetes.io/kube-apiserver-client-kubelet.UnhandledError" Sep 10 13:18:10.392067 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.391955 2560 manager.go:216] Machine: {Timestamp:2026-09-10 13:18:10.390616906 +0000 UTC m=+0.285334315 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3113500 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27054b1a52f6da6288198827449342 SystemUUID:ec27054b-1a52-f6da-6288-198827449342 BootID:2146eb6d-5423-46b1-a8f4-73355f6ccdaa Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:dc:4b:55:0e:7b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:dc:4b:55:0e:7b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d6:90:7c:1f:69:8a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 10 13:18:10.392067 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.392060 2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 10 13:18:10.392244 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.392227 2560 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 10 13:18:10.395033 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.395009 2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 10 13:18:10.395179 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.395036 2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-118.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 10 13:18:10.396602 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.396591 2560 topology_manager.go:138] "Creating topology manager with none policy" Sep 10 13:18:10.396636 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.396605 2560 container_manager_linux.go:309] "Creating device plugin manager" Sep 10 13:18:10.396636 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.396613 2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 13:18:10.397142 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.397132 2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 13:18:10.398434 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.398423 2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 10 13:18:10.401107 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.401095 2560 state_mem.go:36] "Initialized new in-memory state store" Sep 10 13:18:10.401228 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.401219 2560 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 10 13:18:10.401271 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.401262 2560 kubelet.go:486] "Attempting to sync node with API server" Sep 10 13:18:10.401313 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.401289 2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 10 13:18:10.401350 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.401314 2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 10 13:18:10.401350 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.401325 2560 kubelet.go:398] "Adding apiserver pod source" Sep 10 13:18:10.401350 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.401334 2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 10 13:18:10.401708 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.401697 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 13:18:10.401756 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.401715 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 13:18:10.404936 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.404909 2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 10 13:18:10.407021 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.407004 2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 10 13:18:10.407086 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.407035 2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 10 13:18:10.407218 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.407209 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 10 13:18:10.407259 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.407223 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 10 13:18:10.407259 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.407229 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 10 13:18:10.407259 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.407235 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 10 13:18:10.407259 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.407242 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 10 13:18:10.407259 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.407248 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 10 13:18:10.407509 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.407500 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 10 13:18:10.407509 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.407510 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 10 13:18:10.407572 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.407517 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 10 13:18:10.407572 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.407523 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 10 13:18:10.407572 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.407532 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 10 13:18:10.407572 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.407540 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 10 13:18:10.408301 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.408292 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 10 13:18:10.408301 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.408301 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 10 13:18:10.411465 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.411430 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 10 13:18:10.411629 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.411610 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 10 13:18:10.411678 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.411626 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:10.412716 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.412702 2560 server.go:1268] "Started kubelet" Sep 10 13:18:10.412792 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.412753 2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 10 13:18:10.412888 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.412852 2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 10 13:18:10.412991 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.412904 2560 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 10 13:18:10.413837 ip-10-0-142-118 systemd[1]: Started Kubernetes Kubelet. Sep 10 13:18:10.414573 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.414557 2560 server.go:310] "Adding debug handlers to kubelet server" Sep 10 13:18:10.415104 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.415084 2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 10 13:18:10.416968 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.416947 2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 10 13:18:10.417425 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.417391 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 10 13:18:10.417889 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.417871 2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 10 13:18:10.418077 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.418064 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:18:10.418666 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.418647 2560 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 10 13:18:10.418666 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.418658 2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 10 13:18:10.418666 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.418667 2560 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 10 13:18:10.418841 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.418768 2560 reconstruct.go:97] "Volume reconstruction finished" Sep 10 13:18:10.418841 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.418777 2560 reconciler.go:29] "Reconciler: start to sync state" Sep 10 13:18:10.418841 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.418792 2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 10 13:18:10.418841 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.418807 2560 factory.go:55] Registering systemd factory Sep 10 13:18:10.418841 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.418817 2560 factory.go:223] Registration of the systemd container factory successfully Sep 10 13:18:10.419108 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.419063 2560 factory.go:153] Registering CRI-O factory Sep 10 13:18:10.419108 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.419076 2560 factory.go:223] Registration of the crio container factory successfully Sep 10 13:18:10.419108 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.419092 2560 factory.go:103] Registering Raw factory Sep 10 13:18:10.419108 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.419100 2560 manager.go:1196] Started watching for new ooms in manager Sep 10 13:18:10.419414 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.419386 2560 manager.go:349] Starting recovery of all containers Sep 10 13:18:10.424956 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.424916 2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 10 13:18:10.424956 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.424935 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 10 13:18:10.425658 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.424769 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-118.ec2.internal.18d3f81f52c717e5 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-118.ec2.internal,UID:ip-10-0-142-118.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:10.412672997 +0000 UTC m=+0.307390406,LastTimestamp:2026-09-10 13:18:10.412672997 +0000 UTC m=+0.307390406,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:10.430613 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.430594 2560 manager.go:354] Recovery completed Sep 10 13:18:10.434940 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.434926 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:18:10.436762 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.436740 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:18:10.436819 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.436777 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:18:10.436819 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.436787 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:18:10.437383 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.437368 2560 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 10 13:18:10.437383 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.437374 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:18:10.437383 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.437379 2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 10 13:18:10.437576 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.437394 2560 state_mem.go:36] "Initialized new in-memory state store" Sep 10 13:18:10.438033 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.438015 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:18:10.438112 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.438042 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:18:10.438112 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.438056 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:18:10.438637 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.438626 2560 policy_none.go:49] "None policy: Start" Sep 10 13:18:10.438672 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.438640 2560 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 10 13:18:10.438672 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.438650 2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 10 13:18:10.439104 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.439017 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-118.ec2.internal.18d3f81f5436a96a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-118.ec2.internal,UID:ip-10-0-142-118.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-118.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:10.436761962 +0000 UTC m=+0.331479371,LastTimestamp:2026-09-10 13:18:10.436761962 +0000 UTC m=+0.331479371,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:10.440724 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.440710 2560 policy_none.go:47] "Start" Sep 10 13:18:10.450550 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.450450 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-118.ec2.internal.18d3f81f5436f623 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-118.ec2.internal,UID:ip-10-0-142-118.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-142-118.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:10.436781603 +0000 UTC m=+0.331499012,LastTimestamp:2026-09-10 13:18:10.436781603 +0000 UTC m=+0.331499012,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:10.458584 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.458512 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-118.ec2.internal.18d3f81f54371aa7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-118.ec2.internal,UID:ip-10-0-142-118.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-142-118.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:10.436790951 +0000 UTC m=+0.331508360,LastTimestamp:2026-09-10 13:18:10.436790951 +0000 UTC m=+0.331508360,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:10.468211 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.468185 2560 manager.go:337] "Starting Device Plugin manager" Sep 10 13:18:10.483097 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.468233 2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 10 13:18:10.483097 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.468247 2560 server.go:87] "Starting device plugin registration server" Sep 10 13:18:10.483097 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.468229 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-142-118.ec2.internal.18d3f81f5436a96a\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-118.ec2.internal.18d3f81f5436a96a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-118.ec2.internal,UID:ip-10-0-142-118.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-118.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:10.436761962 +0000 UTC m=+0.331479371,LastTimestamp:2026-09-10 13:18:10.438030398 +0000 UTC m=+0.332747806,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:10.483097 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.468555 2560 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 10 13:18:10.483097 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.468567 2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 10 13:18:10.483097 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.468687 2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 10 13:18:10.483097 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.468773 2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 10 13:18:10.483097 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.468783 2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 10 13:18:10.483097 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.469432 2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 10 13:18:10.483097 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.469479 2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:18:10.483097 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.480095 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-142-118.ec2.internal.18d3f81f5436f623\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-118.ec2.internal.18d3f81f5436f623 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-118.ec2.internal,UID:ip-10-0-142-118.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-142-118.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:10.436781603 +0000 UTC m=+0.331499012,LastTimestamp:2026-09-10 13:18:10.438048323 +0000 UTC m=+0.332765733,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:10.490149 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.490074 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-142-118.ec2.internal.18d3f81f54371aa7\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-118.ec2.internal.18d3f81f54371aa7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-118.ec2.internal,UID:ip-10-0-142-118.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-142-118.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:10.436790951 +0000 UTC m=+0.331508360,LastTimestamp:2026-09-10 13:18:10.438062515 +0000 UTC m=+0.332779923,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:10.499342 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.499275 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-118.ec2.internal.18d3f81f56a985a0 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-118.ec2.internal,UID:ip-10-0-142-118.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:10.477843872 +0000 UTC m=+0.372561271,LastTimestamp:2026-09-10 13:18:10.477843872 +0000 UTC m=+0.372561271,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:10.554858 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.554814 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 10 13:18:10.556089 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.556073 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 10 13:18:10.556168 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.556135 2560 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 10 13:18:10.556168 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.556163 2560 kubelet.go:2448] "Starting kubelet main sync loop" Sep 10 13:18:10.556273 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.556207 2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 10 13:18:10.566213 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.566185 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" Sep 10 13:18:10.569308 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.569283 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:18:10.570151 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.570134 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:18:10.570257 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.570171 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:18:10.570257 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.570185 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:18:10.570257 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.570213 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-118.ec2.internal" Sep 10 13:18:10.581524 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.581431 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-142-118.ec2.internal.18d3f81f5436a96a\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-118.ec2.internal.18d3f81f5436a96a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-118.ec2.internal,UID:ip-10-0-142-118.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-118.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:10.436761962 +0000 UTC m=+0.331479371,LastTimestamp:2026-09-10 13:18:10.570151645 +0000 UTC m=+0.464869061,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:10.590689 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.590661 2560 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-142-118.ec2.internal" Sep 10 13:18:10.590919 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.590853 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-142-118.ec2.internal.18d3f81f5436f623\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-118.ec2.internal.18d3f81f5436f623 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-118.ec2.internal,UID:ip-10-0-142-118.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-142-118.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:10.436781603 +0000 UTC m=+0.331499012,LastTimestamp:2026-09-10 13:18:10.570178079 +0000 UTC m=+0.464895495,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:10.600847 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.600719 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-142-118.ec2.internal.18d3f81f54371aa7\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-118.ec2.internal.18d3f81f54371aa7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-118.ec2.internal,UID:ip-10-0-142-118.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-142-118.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:10.436790951 +0000 UTC m=+0.331508360,LastTimestamp:2026-09-10 13:18:10.570190762 +0000 UTC m=+0.464908173,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:10.634030 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.633991 2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="400ms" Sep 10 13:18:10.657216 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.657175 2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-118.ec2.internal"] Sep 10 13:18:10.657296 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.657267 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:18:10.658174 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.658156 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:18:10.658247 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.658187 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:18:10.658247 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.658197 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:18:10.659266 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.659253 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:18:10.659373 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.659359 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal" Sep 10 13:18:10.659434 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.659388 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:18:10.659951 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.659938 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:18:10.659951 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.659945 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:18:10.660071 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.659966 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:18:10.660071 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.659972 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:18:10.660071 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.659982 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:18:10.660071 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.659983 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:18:10.661029 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.661015 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-118.ec2.internal" Sep 10 13:18:10.661077 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.661044 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:18:10.661702 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.661658 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:18:10.661762 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.661713 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:18:10.661762 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.661722 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:18:10.671459 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.671371 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-142-118.ec2.internal.18d3f81f5436a96a\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-118.ec2.internal.18d3f81f5436a96a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-118.ec2.internal,UID:ip-10-0-142-118.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-118.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:10.436761962 +0000 UTC m=+0.331479371,LastTimestamp:2026-09-10 13:18:10.658173309 +0000 UTC m=+0.552890718,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:10.685116 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.685041 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-142-118.ec2.internal.18d3f81f5436f623\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-118.ec2.internal.18d3f81f5436f623 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-118.ec2.internal,UID:ip-10-0-142-118.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-142-118.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:10.436781603 +0000 UTC m=+0.331499012,LastTimestamp:2026-09-10 13:18:10.658191933 +0000 UTC m=+0.552909342,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:10.690274 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.690253 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-118.ec2.internal\" not found" node="ip-10-0-142-118.ec2.internal" Sep 10 13:18:10.694718 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.694641 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-142-118.ec2.internal.18d3f81f54371aa7\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-118.ec2.internal.18d3f81f54371aa7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-118.ec2.internal,UID:ip-10-0-142-118.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-142-118.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:10.436790951 +0000 UTC m=+0.331508360,LastTimestamp:2026-09-10 13:18:10.65820131 +0000 UTC m=+0.552918720,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:10.694867 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.694851 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-118.ec2.internal\" not found" node="ip-10-0-142-118.ec2.internal" Sep 10 13:18:10.705126 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.703850 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-142-118.ec2.internal.18d3f81f5436a96a\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-118.ec2.internal.18d3f81f5436a96a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-118.ec2.internal,UID:ip-10-0-142-118.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-118.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:10.436761962 +0000 UTC m=+0.331479371,LastTimestamp:2026-09-10 13:18:10.65995092 +0000 UTC m=+0.554668334,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:10.713889 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.713827 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-142-118.ec2.internal.18d3f81f5436a96a\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-118.ec2.internal.18d3f81f5436a96a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-118.ec2.internal,UID:ip-10-0-142-118.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-118.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:10.436761962 +0000 UTC m=+0.331479371,LastTimestamp:2026-09-10 13:18:10.659960043 +0000 UTC m=+0.554677456,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:10.720384 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.720367 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b43f7a99705be5e3239e1140886f9e8c-config\") pod \"kube-apiserver-proxy-ip-10-0-142-118.ec2.internal\" (UID: \"b43f7a99705be5e3239e1140886f9e8c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-118.ec2.internal" Sep 10 13:18:10.720466 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.720392 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/244c08a5130c7e774ed4909b145d575a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal\" (UID: \"244c08a5130c7e774ed4909b145d575a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal" Sep 10 13:18:10.720466 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.720418 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/244c08a5130c7e774ed4909b145d575a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal\" (UID: \"244c08a5130c7e774ed4909b145d575a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal" Sep 10 13:18:10.723118 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.723061 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-142-118.ec2.internal.18d3f81f5436f623\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-118.ec2.internal.18d3f81f5436f623 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-118.ec2.internal,UID:ip-10-0-142-118.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-142-118.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:10.436781603 +0000 UTC m=+0.331499012,LastTimestamp:2026-09-10 13:18:10.65997251 +0000 UTC m=+0.554689924,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:10.732497 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.732439 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-142-118.ec2.internal.18d3f81f5436f623\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-118.ec2.internal.18d3f81f5436f623 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-118.ec2.internal,UID:ip-10-0-142-118.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-142-118.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:10.436781603 +0000 UTC m=+0.331499012,LastTimestamp:2026-09-10 13:18:10.65997785 +0000 UTC m=+0.554695259,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:10.743410 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.743336 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-142-118.ec2.internal.18d3f81f54371aa7\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-118.ec2.internal.18d3f81f54371aa7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-118.ec2.internal,UID:ip-10-0-142-118.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-142-118.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:10.436790951 +0000 UTC m=+0.331508360,LastTimestamp:2026-09-10 13:18:10.659989032 +0000 UTC m=+0.554706442,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:10.752731 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.752664 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-142-118.ec2.internal.18d3f81f54371aa7\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-118.ec2.internal.18d3f81f54371aa7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-118.ec2.internal,UID:ip-10-0-142-118.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-142-118.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:10.436790951 +0000 UTC m=+0.331508360,LastTimestamp:2026-09-10 13:18:10.660045394 +0000 UTC m=+0.554762835,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:10.763183 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.763109 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-142-118.ec2.internal.18d3f81f5436a96a\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-118.ec2.internal.18d3f81f5436a96a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-118.ec2.internal,UID:ip-10-0-142-118.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-118.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:10.436761962 +0000 UTC m=+0.331479371,LastTimestamp:2026-09-10 13:18:10.661700863 +0000 UTC m=+0.556418274,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:10.773103 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.773027 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-142-118.ec2.internal.18d3f81f5436f623\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-118.ec2.internal.18d3f81f5436f623 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-118.ec2.internal,UID:ip-10-0-142-118.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-142-118.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:10.436781603 +0000 UTC m=+0.331499012,LastTimestamp:2026-09-10 13:18:10.661717491 +0000 UTC m=+0.556434899,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:10.783539 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.783462 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-142-118.ec2.internal.18d3f81f54371aa7\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-118.ec2.internal.18d3f81f54371aa7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-118.ec2.internal,UID:ip-10-0-142-118.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-142-118.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:10.436790951 +0000 UTC m=+0.331508360,LastTimestamp:2026-09-10 13:18:10.661725833 +0000 UTC m=+0.556443242,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:10.791612 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.791591 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:18:10.792552 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.792532 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:18:10.792650 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.792571 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:18:10.792650 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.792586 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:18:10.792650 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.792621 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-118.ec2.internal" Sep 10 13:18:10.806093 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.806000 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-142-118.ec2.internal.18d3f81f5436a96a\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-118.ec2.internal.18d3f81f5436a96a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-118.ec2.internal,UID:ip-10-0-142-118.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-118.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:10.436761962 +0000 UTC m=+0.331479371,LastTimestamp:2026-09-10 13:18:10.792552742 +0000 UTC m=+0.687270158,Count:8,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:10.814963 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.814934 2560 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-142-118.ec2.internal" Sep 10 13:18:10.814963 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:10.814908 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-142-118.ec2.internal.18d3f81f5436f623\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-118.ec2.internal.18d3f81f5436f623 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-118.ec2.internal,UID:ip-10-0-142-118.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-142-118.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:10.436781603 +0000 UTC m=+0.331499012,LastTimestamp:2026-09-10 13:18:10.792577531 +0000 UTC m=+0.687294941,Count:8,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:10.821141 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.821122 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b43f7a99705be5e3239e1140886f9e8c-config\") pod \"kube-apiserver-proxy-ip-10-0-142-118.ec2.internal\" (UID: \"b43f7a99705be5e3239e1140886f9e8c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-118.ec2.internal" Sep 10 13:18:10.821206 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.821147 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/244c08a5130c7e774ed4909b145d575a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal\" (UID: \"244c08a5130c7e774ed4909b145d575a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal" Sep 10 13:18:10.821206 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.821162 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/244c08a5130c7e774ed4909b145d575a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal\" (UID: \"244c08a5130c7e774ed4909b145d575a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal" Sep 10 13:18:10.821206 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.821205 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/244c08a5130c7e774ed4909b145d575a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal\" (UID: \"244c08a5130c7e774ed4909b145d575a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal" Sep 10 13:18:10.821303 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.821216 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b43f7a99705be5e3239e1140886f9e8c-config\") pod \"kube-apiserver-proxy-ip-10-0-142-118.ec2.internal\" (UID: \"b43f7a99705be5e3239e1140886f9e8c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-118.ec2.internal" Sep 10 13:18:10.821303 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.821217 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/244c08a5130c7e774ed4909b145d575a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal\" (UID: \"244c08a5130c7e774ed4909b145d575a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal" Sep 10 13:18:10.991992 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.991927 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal" Sep 10 13:18:10.997727 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:10.997709 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-118.ec2.internal" Sep 10 13:18:11.046073 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:11.046037 2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="800ms" Sep 10 13:18:11.216463 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:11.216426 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:18:11.217534 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:11.217517 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:18:11.217609 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:11.217550 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:18:11.217609 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:11.217562 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:18:11.217609 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:11.217599 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-118.ec2.internal" Sep 10 13:18:11.236869 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:11.236842 2560 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-142-118.ec2.internal" Sep 10 13:18:11.357001 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:11.356920 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 10 13:18:11.425344 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:11.425309 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:11.496035 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:11.495988 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:18:11.497644 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:11.497623 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:18:11.497772 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:11.497669 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:18:11.497772 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:11.497680 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:18:11.587625 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:11.587445 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb43f7a99705be5e3239e1140886f9e8c.slice/crio-7203e8b5765ea211a02e3a04a144145afb684c4d08b7acaab3a53dbd29695862 WatchSource:0}: Error finding container 7203e8b5765ea211a02e3a04a144145afb684c4d08b7acaab3a53dbd29695862: Status 404 returned error can't find the container with id 7203e8b5765ea211a02e3a04a144145afb684c4d08b7acaab3a53dbd29695862 Sep 10 13:18:11.593058 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:11.593040 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 13:18:11.603103 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:11.603015 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-142-118.ec2.internal.18d3f81f99258617 kube-system 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-142-118.ec2.internal,UID:b43f7a99705be5e3239e1140886f9e8c,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Pulling,Message:Pulling image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:da130d90f0bffd5c5876c94544c91b2a1be0d23ce2747ddfeafe88e03ee7d8d1\",Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:11.593266711 +0000 UTC m=+1.487984108,LastTimestamp:2026-09-10 13:18:11.593266711 +0000 UTC m=+1.487984108,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:11.735560 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:11.735470 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 10 13:18:11.811468 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:11.811436 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" Sep 10 13:18:11.841326 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:11.841294 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 10 13:18:11.856695 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:11.856660 2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="1.6s" Sep 10 13:18:11.910894 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:18:11.910859 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod244c08a5130c7e774ed4909b145d575a.slice/crio-26ad1edcb6fbc4feef3f6a5de973c50c88a37b9937dd4923725a7beb382e635e WatchSource:0}: Error finding container 26ad1edcb6fbc4feef3f6a5de973c50c88a37b9937dd4923725a7beb382e635e: Status 404 returned error can't find the container with id 26ad1edcb6fbc4feef3f6a5de973c50c88a37b9937dd4923725a7beb382e635e Sep 10 13:18:11.925112 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:11.925028 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal.18d3f81fac3069e9 openshift-machine-config-operator 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal,UID:244c08a5130c7e774ed4909b145d575a,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulling,Message:Pulling image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cec847b1f08cae0315bd44fe477d725a0180b78ab0ad844de3b218bf2dde0bb2\",Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:11.912747497 +0000 UTC m=+1.807464895,LastTimestamp:2026-09-10 13:18:11.912747497 +0000 UTC m=+1.807464895,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:12.038004 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:12.037901 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:18:12.039050 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:12.039031 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:18:12.039127 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:12.039066 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:18:12.039127 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:12.039081 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:18:12.039127 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:12.039113 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-118.ec2.internal" Sep 10 13:18:12.066222 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:12.066189 2560 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-142-118.ec2.internal" Sep 10 13:18:12.424463 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:12.424364 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:12.563032 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:12.562893 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal" event={"ID":"244c08a5130c7e774ed4909b145d575a","Type":"ContainerStarted","Data":"26ad1edcb6fbc4feef3f6a5de973c50c88a37b9937dd4923725a7beb382e635e"} Sep 10 13:18:12.564548 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:12.564520 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-118.ec2.internal" event={"ID":"b43f7a99705be5e3239e1140886f9e8c","Type":"ContainerStarted","Data":"7203e8b5765ea211a02e3a04a144145afb684c4d08b7acaab3a53dbd29695862"} Sep 10 13:18:12.572791 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:12.572703 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 13:18:12.576204 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:12.576171 2560 certificate_manager.go:596] "Failed while requesting a signed certificate from the control plane" err="cannot create certificate signing request: certificatesigningrequests.certificates.k8s.io is forbidden: User \"system:serviceaccount:openshift-machine-config-operator:node-bootstrapper\" cannot create resource \"certificatesigningrequests\" in API group \"certificates.k8s.io\" at the cluster scope" logger="kubernetes.io/kube-apiserver-client-kubelet.UnhandledError" Sep 10 13:18:13.426385 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:13.426345 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:13.468580 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:13.468541 2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="3.2s" Sep 10 13:18:13.667225 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:13.667188 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:18:13.668293 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:13.668270 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:18:13.668421 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:13.668314 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:18:13.668421 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:13.668332 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:18:13.668421 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:13.668366 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-118.ec2.internal" Sep 10 13:18:13.698526 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:13.698444 2560 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-142-118.ec2.internal" Sep 10 13:18:13.764025 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:13.763986 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:18:13.765207 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:13.765182 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:18:13.765326 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:13.765226 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:18:13.765326 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:13.765243 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:18:13.778452 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:13.778411 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 10 13:18:13.845252 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:13.845215 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 10 13:18:13.968383 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:13.968298 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal.18d3f82025d57de9 openshift-machine-config-operator 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal,UID:244c08a5130c7e774ed4909b145d575a,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulled,Message:Successfully pulled image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cec847b1f08cae0315bd44fe477d725a0180b78ab0ad844de3b218bf2dde0bb2\" in 2.04s (2.04s including waiting). Image size: 282837706 bytes.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:13.953609193 +0000 UTC m=+3.848326605,LastTimestamp:2026-09-10 13:18:13.953609193 +0000 UTC m=+3.848326605,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:13.980042 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:13.979951 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-142-118.ec2.internal.18d3f82025d83f60 kube-system 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-142-118.ec2.internal,UID:b43f7a99705be5e3239e1140886f9e8c,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Pulled,Message:Successfully pulled image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:da130d90f0bffd5c5876c94544c91b2a1be0d23ce2747ddfeafe88e03ee7d8d1\" in 2.36s (2.36s including waiting). Image size: 321671430 bytes.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:13.953789792 +0000 UTC m=+3.848507207,LastTimestamp:2026-09-10 13:18:13.953789792 +0000 UTC m=+3.848507207,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:13.980042 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:13.980035 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 10 13:18:14.031155 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:14.031068 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-142-118.ec2.internal.18d3f82029980f8b kube-system 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-142-118.ec2.internal,UID:b43f7a99705be5e3239e1140886f9e8c,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Created,Message:Created container: haproxy,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:14.016692107 +0000 UTC m=+3.911409520,LastTimestamp:2026-09-10 13:18:14.016692107 +0000 UTC m=+3.911409520,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:14.042810 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:14.042708 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-142-118.ec2.internal.18d3f8202a07f394 kube-system 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-142-118.ec2.internal,UID:b43f7a99705be5e3239e1140886f9e8c,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Started,Message:Started container haproxy,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:14.02402498 +0000 UTC m=+3.918742402,LastTimestamp:2026-09-10 13:18:14.02402498 +0000 UTC m=+3.918742402,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:14.174751 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:14.174664 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal.18d3f8203256e7f6 openshift-machine-config-operator 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal,UID:244c08a5130c7e774ed4909b145d575a,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Created,Message:Created container: setup,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:14.163417078 +0000 UTC m=+4.058134499,LastTimestamp:2026-09-10 13:18:14.163417078 +0000 UTC m=+4.058134499,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:14.185591 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:14.185514 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal.18d3f82032d6445d openshift-machine-config-operator 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal,UID:244c08a5130c7e774ed4909b145d575a,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Started,Message:Started container setup,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:14.171763805 +0000 UTC m=+4.066481222,LastTimestamp:2026-09-10 13:18:14.171763805 +0000 UTC m=+4.066481222,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:14.234173 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:14.234099 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" Sep 10 13:18:14.422316 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:14.422289 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:14.569091 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:14.568996 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal" event={"ID":"244c08a5130c7e774ed4909b145d575a","Type":"ContainerStarted","Data":"5b615caeb1e75829b26704dd0281b369271cd126ecd9e641919f7d3e5ea3a039"} Sep 10 13:18:14.569091 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:14.569030 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:18:14.570238 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:14.570219 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:18:14.570357 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:14.570248 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:18:14.570357 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:14.570262 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:18:14.570357 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:14.570307 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-118.ec2.internal" event={"ID":"b43f7a99705be5e3239e1140886f9e8c","Type":"ContainerStarted","Data":"71d9e5cf8b5f8bacfdc86a425bd094f63a4f7eb34eb1d881627d51defbb71bb6"} Sep 10 13:18:14.570524 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:14.570470 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-118.ec2.internal\" not found" node="ip-10-0-142-118.ec2.internal" Sep 10 13:18:14.570524 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:14.570476 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:18:14.571116 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:14.571100 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:18:14.571206 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:14.571128 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:18:14.571206 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:14.571142 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:18:14.571335 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:14.571320 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-118.ec2.internal\" not found" node="ip-10-0-142-118.ec2.internal" Sep 10 13:18:15.429276 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:15.429195 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:15.571562 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:15.571536 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:18:15.571888 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:15.571536 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:18:15.572386 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:15.572368 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:18:15.572486 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:15.572382 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:18:15.572486 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:15.572418 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:18:15.572486 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:15.572423 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:18:15.572486 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:15.572431 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:18:15.572486 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:15.572437 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:18:15.572663 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:15.572649 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-118.ec2.internal\" not found" node="ip-10-0-142-118.ec2.internal" Sep 10 13:18:15.572825 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:15.572810 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-118.ec2.internal\" not found" node="ip-10-0-142-118.ec2.internal" Sep 10 13:18:16.422935 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:16.422903 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:16.684167 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:16.684090 2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="6.4s" Sep 10 13:18:16.899175 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:16.899126 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:18:16.900120 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:16.900102 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:18:16.900228 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:16.900130 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:18:16.900228 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:16.900140 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:18:16.900228 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:16.900166 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-118.ec2.internal" Sep 10 13:18:16.918380 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:16.918350 2560 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-142-118.ec2.internal" Sep 10 13:18:16.924504 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:16.924487 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 13:18:16.927166 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:16.927142 2560 certificate_manager.go:596] "Failed while requesting a signed certificate from the control plane" err="cannot create certificate signing request: certificatesigningrequests.certificates.k8s.io is forbidden: User \"system:serviceaccount:openshift-machine-config-operator:node-bootstrapper\" cannot create resource \"certificatesigningrequests\" in API group \"certificates.k8s.io\" at the cluster scope" logger="kubernetes.io/kube-apiserver-client-kubelet.UnhandledError" Sep 10 13:18:17.422070 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:17.422039 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:17.698087 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:17.698003 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" Sep 10 13:18:17.727414 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:17.727364 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 10 13:18:18.420990 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:18.420956 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:18.477201 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:18.477167 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 10 13:18:18.559715 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:18.559679 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:18:18.560677 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:18.560660 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:18:18.560741 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:18.560696 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:18:18.560741 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:18.560709 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:18:19.421690 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:19.421661 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:19.807728 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:19.807701 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 10 13:18:20.421531 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:20.421504 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:20.469827 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:20.469786 2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:18:21.422001 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:21.421968 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:22.422719 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:22.422687 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:23.092824 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:23.092786 2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s" Sep 10 13:18:23.319141 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:23.319095 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:18:23.320068 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:23.320048 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:18:23.320172 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:23.320085 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:18:23.320172 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:23.320103 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:18:23.320172 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:23.320134 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-118.ec2.internal" Sep 10 13:18:23.339439 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:23.339391 2560 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-142-118.ec2.internal" Sep 10 13:18:23.420961 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:23.420887 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:24.421259 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:24.421225 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:25.422533 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:25.422504 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:25.586369 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:25.586298 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal_244c08a5130c7e774ed4909b145d575a/setup/0.log" Sep 10 13:18:25.586369 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:25.586336 2560 generic.go:356] "Generic (PLEG): container finished" podID="244c08a5130c7e774ed4909b145d575a" containerID="5b615caeb1e75829b26704dd0281b369271cd126ecd9e641919f7d3e5ea3a039" exitCode=1 Sep 10 13:18:25.586559 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:25.586372 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal" event={"ID":"244c08a5130c7e774ed4909b145d575a","Type":"ContainerDied","Data":"5b615caeb1e75829b26704dd0281b369271cd126ecd9e641919f7d3e5ea3a039"} Sep 10 13:18:25.586559 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:25.586483 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:18:25.587277 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:25.587253 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:18:25.587378 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:25.587285 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:18:25.587378 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:25.587294 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:18:25.587510 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:25.587497 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-118.ec2.internal\" not found" node="ip-10-0-142-118.ec2.internal" Sep 10 13:18:25.599450 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:25.599361 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal.18d3f822db6d4a76 openshift-machine-config-operator 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal,UID:244c08a5130c7e774ed4909b145d575a,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulled,Message:Container image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cec847b1f08cae0315bd44fe477d725a0180b78ab0ad844de3b218bf2dde0bb2\" already present on machine,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:25.590168182 +0000 UTC m=+15.484885594,LastTimestamp:2026-09-10 13:18:25.590168182 +0000 UTC m=+15.484885594,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:25.676512 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:25.676491 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 13:18:25.680944 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:25.680923 2560 certificate_manager.go:596] "Failed while requesting a signed certificate from the control plane" err="cannot create certificate signing request: certificatesigningrequests.certificates.k8s.io is forbidden: User \"system:serviceaccount:openshift-machine-config-operator:node-bootstrapper\" cannot create resource \"certificatesigningrequests\" in API group \"certificates.k8s.io\" at the cluster scope" logger="kubernetes.io/kube-apiserver-client-kubelet.UnhandledError" Sep 10 13:18:25.683153 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:25.683077 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal.18d3f8203256e7f6\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal.18d3f8203256e7f6 openshift-machine-config-operator 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal,UID:244c08a5130c7e774ed4909b145d575a,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Created,Message:Created container: setup,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:14.163417078 +0000 UTC m=+4.058134499,LastTimestamp:2026-09-10 13:18:25.671692745 +0000 UTC m=+15.566410166,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:25.693124 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:25.693056 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal.18d3f82032d6445d\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal.18d3f82032d6445d openshift-machine-config-operator 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal,UID:244c08a5130c7e774ed4909b145d575a,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Started,Message:Started container setup,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:14.171763805 +0000 UTC m=+4.066481222,LastTimestamp:2026-09-10 13:18:25.679107365 +0000 UTC m=+15.573824775,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:26.421496 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:26.421461 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:26.591304 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:26.591276 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal_244c08a5130c7e774ed4909b145d575a/setup/0.log" Sep 10 13:18:26.591688 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:26.591324 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal" event={"ID":"244c08a5130c7e774ed4909b145d575a","Type":"ContainerStarted","Data":"4d7ebb119f645bd53d1cac12d2387e667f0361d5b4ea39d25d6634a8acaeba2e"} Sep 10 13:18:26.591688 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:26.591465 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:18:26.592231 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:26.592211 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:18:26.592337 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:26.592244 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:18:26.592337 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:26.592257 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:18:26.592439 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:26.592431 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-118.ec2.internal\" not found" node="ip-10-0-142-118.ec2.internal" Sep 10 13:18:27.420931 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:27.420897 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:28.021562 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:28.021527 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 10 13:18:28.151532 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:28.151489 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:18:28.152662 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:28.152641 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:18:28.152748 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:28.152674 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:18:28.152748 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:28.152685 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:18:28.422095 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:28.422035 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:29.247637 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:29.247605 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 10 13:18:29.422021 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:29.421987 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:30.102688 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:30.102648 2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s" Sep 10 13:18:30.339534 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:30.339493 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:18:30.340577 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:30.340559 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:18:30.340680 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:30.340590 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:18:30.340680 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:30.340600 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:18:30.340680 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:30.340626 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-118.ec2.internal" Sep 10 13:18:30.362768 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:30.362693 2560 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-142-118.ec2.internal" Sep 10 13:18:30.362867 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:30.362777 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" Sep 10 13:18:30.423132 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:30.423103 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:30.423283 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:30.423150 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 10 13:18:30.470436 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:30.470385 2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:18:31.422603 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:31.422572 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:32.421728 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:32.421701 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:33.422172 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:33.422144 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:34.422915 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:34.422885 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:35.421022 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:35.420991 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:36.421867 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:36.421841 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:37.113225 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:37.113117 2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s" Sep 10 13:18:37.363371 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:37.363285 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:18:37.364349 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:37.364328 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:18:37.364475 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:37.364366 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:18:37.364475 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:37.364380 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:18:37.364475 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:37.364427 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-118.ec2.internal" Sep 10 13:18:37.380871 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:37.380845 2560 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-142-118.ec2.internal" Sep 10 13:18:37.421804 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:37.421779 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:37.610518 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:37.610490 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal_244c08a5130c7e774ed4909b145d575a/setup/1.log" Sep 10 13:18:37.610949 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:37.610897 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal_244c08a5130c7e774ed4909b145d575a/setup/0.log" Sep 10 13:18:37.610994 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:37.610927 2560 generic.go:356] "Generic (PLEG): container finished" podID="244c08a5130c7e774ed4909b145d575a" containerID="4d7ebb119f645bd53d1cac12d2387e667f0361d5b4ea39d25d6634a8acaeba2e" exitCode=1 Sep 10 13:18:37.610994 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:37.610982 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal" event={"ID":"244c08a5130c7e774ed4909b145d575a","Type":"ContainerDied","Data":"4d7ebb119f645bd53d1cac12d2387e667f0361d5b4ea39d25d6634a8acaeba2e"} Sep 10 13:18:37.611054 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:37.611025 2560 scope.go:117] "RemoveContainer" containerID="5b615caeb1e75829b26704dd0281b369271cd126ecd9e641919f7d3e5ea3a039" Sep 10 13:18:37.611113 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:37.611098 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:18:37.611874 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:37.611857 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:18:37.611984 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:37.611892 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:18:37.611984 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:37.611904 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:18:37.612502 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:37.612073 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-118.ec2.internal\" not found" node="ip-10-0-142-118.ec2.internal" Sep 10 13:18:37.612502 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:37.612120 2560 scope.go:117] "RemoveContainer" containerID="5b615caeb1e75829b26704dd0281b369271cd126ecd9e641919f7d3e5ea3a039" Sep 10 13:18:37.620299 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:37.620274 2560 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal_openshift-machine-config-operator_244c08a5130c7e774ed4909b145d575a_0 in pod sandbox 26ad1edcb6fbc4feef3f6a5de973c50c88a37b9937dd4923725a7beb382e635e from index: no such id: '5b615caeb1e75829b26704dd0281b369271cd126ecd9e641919f7d3e5ea3a039'" containerID="5b615caeb1e75829b26704dd0281b369271cd126ecd9e641919f7d3e5ea3a039" Sep 10 13:18:37.620360 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:37.620318 2560 kuberuntime_container.go:962] "Unhandled Error" err="failed to remove pod init container \"setup\": rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal_openshift-machine-config-operator_244c08a5130c7e774ed4909b145d575a_0 in pod sandbox 26ad1edcb6fbc4feef3f6a5de973c50c88a37b9937dd4923725a7beb382e635e from index: no such id: '5b615caeb1e75829b26704dd0281b369271cd126ecd9e641919f7d3e5ea3a039'; Skipping pod \"kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal_openshift-machine-config-operator(244c08a5130c7e774ed4909b145d575a)\"" logger="UnhandledError" Sep 10 13:18:37.620472 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:37.620456 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal_openshift-machine-config-operator(244c08a5130c7e774ed4909b145d575a)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal" podUID="244c08a5130c7e774ed4909b145d575a" Sep 10 13:18:37.631014 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:37.630941 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal.18d3f825a87c71fb openshift-machine-config-operator 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal,UID:244c08a5130c7e774ed4909b145d575a,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:BackOff,Message:Back-off restarting failed container setup in pod kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal_openshift-machine-config-operator(244c08a5130c7e774ed4909b145d575a),Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:37.620425211 +0000 UTC m=+27.515142619,LastTimestamp:2026-09-10 13:18:37.620425211 +0000 UTC m=+27.515142619,Count:1,Type:Warning,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:38.420414 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:38.420377 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:38.613628 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:38.613600 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal_244c08a5130c7e774ed4909b145d575a/setup/1.log" Sep 10 13:18:39.421900 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:39.421872 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:40.421033 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:40.421004 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:40.471393 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:40.471357 2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:18:41.421195 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:41.421163 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:42.423503 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:42.423474 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:42.676701 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:42.676613 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 13:18:42.679648 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:42.679626 2560 certificate_manager.go:596] "Failed while requesting a signed certificate from the control plane" err="cannot create certificate signing request: certificatesigningrequests.certificates.k8s.io is forbidden: User \"system:serviceaccount:openshift-machine-config-operator:node-bootstrapper\" cannot create resource \"certificatesigningrequests\" in API group \"certificates.k8s.io\" at the cluster scope" logger="kubernetes.io/kube-apiserver-client-kubelet.UnhandledError" Sep 10 13:18:42.679711 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:42.679652 2560 certificate_manager.go:461] "Reached backoff limit, still unable to rotate certs" err="timed out waiting for the condition" logger="kubernetes.io/kube-apiserver-client-kubelet.UnhandledError" Sep 10 13:18:43.424040 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:43.424012 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:44.123392 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:44.123364 2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s" Sep 10 13:18:44.381577 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:44.381508 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:18:44.382525 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:44.382501 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:18:44.382626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:44.382540 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:18:44.382626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:44.382551 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:18:44.382626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:44.382577 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-118.ec2.internal" Sep 10 13:18:44.401214 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:44.401192 2560 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-142-118.ec2.internal" Sep 10 13:18:44.421656 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:44.421542 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:45.422061 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:45.422026 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:46.314019 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:46.313976 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:18:46.314991 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:46.314972 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:18:46.315094 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:46.315001 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:18:46.315094 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:46.315012 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:18:46.422013 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:46.421987 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:47.045352 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:47.045316 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 10 13:18:47.420838 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:47.420807 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:48.421056 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:48.421027 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:49.421623 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:49.421591 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:50.424212 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:50.424180 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:50.472495 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:50.472462 2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:18:51.133344 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:51.133313 2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s" Sep 10 13:18:51.401710 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:51.401634 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:18:51.402648 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:51.402630 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:18:51.402694 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:51.402662 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:18:51.402694 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:51.402674 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:18:51.402758 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:51.402702 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-118.ec2.internal" Sep 10 13:18:51.420939 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:51.420912 2560 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-142-118.ec2.internal" Sep 10 13:18:51.421087 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:51.421029 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:51.556595 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:51.556560 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:18:51.557501 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:51.557485 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:18:51.557555 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:51.557515 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:18:51.557555 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:51.557527 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:18:51.557721 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:51.557709 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-118.ec2.internal\" not found" node="ip-10-0-142-118.ec2.internal" Sep 10 13:18:51.568154 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:51.568061 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal.18d3f822db6d4a76\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal.18d3f822db6d4a76 openshift-machine-config-operator 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal,UID:244c08a5130c7e774ed4909b145d575a,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulled,Message:Container image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cec847b1f08cae0315bd44fe477d725a0180b78ab0ad844de3b218bf2dde0bb2\" already present on machine,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:25.590168182 +0000 UTC m=+15.484885594,LastTimestamp:2026-09-10 13:18:51.558441896 +0000 UTC m=+41.453159296,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:51.652083 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:51.651976 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal.18d3f8203256e7f6\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal.18d3f8203256e7f6 openshift-machine-config-operator 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal,UID:244c08a5130c7e774ed4909b145d575a,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Created,Message:Created container: setup,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:14.163417078 +0000 UTC m=+4.058134499,LastTimestamp:2026-09-10 13:18:51.64138759 +0000 UTC m=+41.536104999,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:51.661825 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:51.661751 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal.18d3f82032d6445d\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal.18d3f82032d6445d openshift-machine-config-operator 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal,UID:244c08a5130c7e774ed4909b145d575a,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Started,Message:Started container setup,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:14.171763805 +0000 UTC m=+4.066481222,LastTimestamp:2026-09-10 13:18:51.648267361 +0000 UTC m=+41.542984780,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:18:51.725854 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:51.725824 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 10 13:18:52.389798 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:52.389764 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 10 13:18:52.431753 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:52.431717 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:52.636606 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:52.636583 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal_244c08a5130c7e774ed4909b145d575a/setup/1.log" Sep 10 13:18:52.636955 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:52.636634 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal" event={"ID":"244c08a5130c7e774ed4909b145d575a","Type":"ContainerStarted","Data":"d4a34d8eeb83ac3e673c74a5da9c728d37c2ee8e27353c6106a034f7f257f2a8"} Sep 10 13:18:52.636955 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:52.636740 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:18:52.637531 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:52.637514 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:18:52.637610 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:52.637544 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:18:52.637610 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:52.637556 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:18:52.637747 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:52.637736 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-118.ec2.internal\" not found" node="ip-10-0-142-118.ec2.internal" Sep 10 13:18:53.421351 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:53.421325 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:54.428023 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:54.427996 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:54.547900 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:54.547870 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" Sep 10 13:18:55.423543 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:55.423503 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:56.423393 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:56.423350 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:57.421753 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:57.421724 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:58.150760 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:58.150726 2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s" Sep 10 13:18:58.421349 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:58.421322 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:18:58.422250 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:58.422228 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:18:58.422360 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:58.422260 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:18:58.422360 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:58.422269 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:18:58.422360 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:58.422296 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-118.ec2.internal" Sep 10 13:18:58.425462 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:58.425442 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:18:58.434209 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:18:58.434190 2560 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-142-118.ec2.internal" Sep 10 13:18:59.415291 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:18:59.415263 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:00.433116 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:00.433086 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:00.473494 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:00.473463 2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:01.444010 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:01.443974 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:02.441901 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:02.441872 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:03.433187 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:03.433151 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:03.654557 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:03.654528 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal_244c08a5130c7e774ed4909b145d575a/setup/2.log" Sep 10 13:19:03.654930 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:03.654912 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal_244c08a5130c7e774ed4909b145d575a/setup/1.log" Sep 10 13:19:03.655040 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:03.654948 2560 generic.go:356] "Generic (PLEG): container finished" podID="244c08a5130c7e774ed4909b145d575a" containerID="d4a34d8eeb83ac3e673c74a5da9c728d37c2ee8e27353c6106a034f7f257f2a8" exitCode=1 Sep 10 13:19:03.655040 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:03.654985 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal" event={"ID":"244c08a5130c7e774ed4909b145d575a","Type":"ContainerDied","Data":"d4a34d8eeb83ac3e673c74a5da9c728d37c2ee8e27353c6106a034f7f257f2a8"} Sep 10 13:19:03.655156 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:03.655081 2560 scope.go:117] "RemoveContainer" containerID="4d7ebb119f645bd53d1cac12d2387e667f0361d5b4ea39d25d6634a8acaeba2e" Sep 10 13:19:03.655207 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:03.655200 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:19:03.656307 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:03.655977 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:19:03.656307 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:03.656019 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:19:03.656307 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:03.656029 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:19:03.656307 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:03.656212 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-118.ec2.internal\" not found" node="ip-10-0-142-118.ec2.internal" Sep 10 13:19:03.656307 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:03.656256 2560 scope.go:117] "RemoveContainer" containerID="4d7ebb119f645bd53d1cac12d2387e667f0361d5b4ea39d25d6634a8acaeba2e" Sep 10 13:19:03.664721 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:03.664687 2560 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal_openshift-machine-config-operator_244c08a5130c7e774ed4909b145d575a_1 in pod sandbox 26ad1edcb6fbc4feef3f6a5de973c50c88a37b9937dd4923725a7beb382e635e from index: no such id: '4d7ebb119f645bd53d1cac12d2387e667f0361d5b4ea39d25d6634a8acaeba2e'" containerID="4d7ebb119f645bd53d1cac12d2387e667f0361d5b4ea39d25d6634a8acaeba2e" Sep 10 13:19:03.664815 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:03.664730 2560 kuberuntime_container.go:962] "Unhandled Error" err="failed to remove pod init container \"setup\": rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal_openshift-machine-config-operator_244c08a5130c7e774ed4909b145d575a_1 in pod sandbox 26ad1edcb6fbc4feef3f6a5de973c50c88a37b9937dd4923725a7beb382e635e from index: no such id: '4d7ebb119f645bd53d1cac12d2387e667f0361d5b4ea39d25d6634a8acaeba2e'; Skipping pod \"kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal_openshift-machine-config-operator(244c08a5130c7e774ed4909b145d575a)\"" logger="UnhandledError" Sep 10 13:19:03.664882 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:03.664859 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal_openshift-machine-config-operator(244c08a5130c7e774ed4909b145d575a)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal" podUID="244c08a5130c7e774ed4909b145d575a" Sep 10 13:19:03.680733 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:03.680644 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal.18d3f825a87c71fb\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal.18d3f825a87c71fb openshift-machine-config-operator 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal,UID:244c08a5130c7e774ed4909b145d575a,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:BackOff,Message:Back-off restarting failed container setup in pod kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal_openshift-machine-config-operator(244c08a5130c7e774ed4909b145d575a),Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:37.620425211 +0000 UTC m=+27.515142619,LastTimestamp:2026-09-10 13:19:03.664826795 +0000 UTC m=+53.559544191,Count:2,Type:Warning,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:19:04.426333 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:04.426298 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:04.657931 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:04.657904 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal_244c08a5130c7e774ed4909b145d575a/setup/2.log" Sep 10 13:19:05.164644 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:05.164607 2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s" Sep 10 13:19:05.429450 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:05.429422 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:05.434527 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:05.434507 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:19:05.435459 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:05.435443 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:19:05.435521 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:05.435473 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:19:05.435521 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:05.435482 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:19:05.435521 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:05.435509 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-118.ec2.internal" Sep 10 13:19:05.453682 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:05.453650 2560 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-142-118.ec2.internal" Sep 10 13:19:06.460822 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:06.460786 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:07.432663 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:07.432628 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:08.426069 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:08.426040 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:09.448420 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:09.448378 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:10.450106 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:10.450078 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:10.474475 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:10.474443 2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:11.432153 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:11.432117 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:12.201760 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:12.201718 2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s" Sep 10 13:19:12.454824 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:12.454725 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:19:12.456391 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:12.456371 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:19:12.456518 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:12.456423 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:19:12.456518 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:12.456439 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:19:12.456518 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:12.456474 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-118.ec2.internal" Sep 10 13:19:12.474684 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:12.474653 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:12.490639 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:12.490608 2560 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-142-118.ec2.internal" Sep 10 13:19:13.417059 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:13.417030 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:14.461223 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:14.461186 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:14.680740 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:14.680716 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 13:19:14.685360 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:14.685337 2560 certificate_manager.go:596] "Failed while requesting a signed certificate from the control plane" err="cannot create certificate signing request: certificatesigningrequests.certificates.k8s.io is forbidden: User \"system:serviceaccount:openshift-machine-config-operator:node-bootstrapper\" cannot create resource \"certificatesigningrequests\" in API group \"certificates.k8s.io\" at the cluster scope" logger="kubernetes.io/kube-apiserver-client-kubelet.UnhandledError" Sep 10 13:19:15.441463 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:15.441433 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:16.441863 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:16.441829 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:17.450179 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:17.450148 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:18.432381 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:18.432348 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:18.557322 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:18.557294 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:19:18.558286 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:18.558269 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:19:18.558391 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:18.558303 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:19:18.558391 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:18.558316 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:19:18.558564 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:18.558548 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-118.ec2.internal\" not found" node="ip-10-0-142-118.ec2.internal" Sep 10 13:19:18.558737 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:18.558717 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal_openshift-machine-config-operator(244c08a5130c7e774ed4909b145d575a)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal" podUID="244c08a5130c7e774ed4909b145d575a" Sep 10 13:19:18.571197 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:18.571109 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal.18d3f825a87c71fb\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal.18d3f825a87c71fb openshift-machine-config-operator 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal,UID:244c08a5130c7e774ed4909b145d575a,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:BackOff,Message:Back-off restarting failed container setup in pod kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal_openshift-machine-config-operator(244c08a5130c7e774ed4909b145d575a),Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:37.620425211 +0000 UTC m=+27.515142619,LastTimestamp:2026-09-10 13:19:18.558687543 +0000 UTC m=+68.453404954,Count:3,Type:Warning,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:19:19.254841 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:19.254807 2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s" Sep 10 13:19:19.422454 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:19.422421 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:19.491355 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:19.491324 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:19:19.492338 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:19.492321 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:19:19.492460 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:19.492355 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:19:19.492460 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:19.492368 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:19:19.492460 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:19.492415 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-118.ec2.internal" Sep 10 13:19:19.531167 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:19.531086 2560 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-142-118.ec2.internal" Sep 10 13:19:20.424076 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:20.424042 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:20.474615 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:20.474572 2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:21.428511 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:21.428481 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:22.431775 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:22.431742 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:23.193483 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:23.193441 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:19:23.194474 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:23.194457 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:19:23.194544 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:23.194489 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:19:23.194544 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:23.194501 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:19:23.461214 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:23.461134 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:24.424571 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:24.424536 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:25.427743 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:25.427718 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:26.288166 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:26.288132 2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s" Sep 10 13:19:26.430105 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:26.430074 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:26.531944 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:26.531919 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:19:26.532990 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:26.532966 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:19:26.533105 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:26.532997 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:19:26.533105 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:26.533007 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:19:26.533105 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:26.533039 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-118.ec2.internal" Sep 10 13:19:26.564808 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:26.564736 2560 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-142-118.ec2.internal" Sep 10 13:19:27.428851 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:27.428821 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:28.424847 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:28.424815 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:29.437548 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:29.437519 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:30.447621 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:30.447592 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:30.474912 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:30.474874 2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:30.556516 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:30.556485 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:19:30.557381 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:30.557357 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:19:30.557381 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:30.557384 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:19:30.557536 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:30.557395 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:19:30.557592 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:30.557580 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-118.ec2.internal\" not found" node="ip-10-0-142-118.ec2.internal" Sep 10 13:19:31.389498 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:31.389462 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 10 13:19:31.431540 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:31.431506 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:31.557179 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:31.557131 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:19:31.558131 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:31.558112 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:19:31.558211 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:31.558145 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:19:31.558211 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:31.558156 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:19:31.558389 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:31.558376 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-118.ec2.internal\" not found" node="ip-10-0-142-118.ec2.internal" Sep 10 13:19:31.571662 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:31.571555 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal.18d3f822db6d4a76\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal.18d3f822db6d4a76 openshift-machine-config-operator 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal,UID:244c08a5130c7e774ed4909b145d575a,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulled,Message:Container image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cec847b1f08cae0315bd44fe477d725a0180b78ab0ad844de3b218bf2dde0bb2\" already present on machine,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:25.590168182 +0000 UTC m=+15.484885594,LastTimestamp:2026-09-10 13:19:31.55914717 +0000 UTC m=+81.453864565,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:19:31.644793 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:31.644685 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal.18d3f8203256e7f6\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal.18d3f8203256e7f6 openshift-machine-config-operator 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal,UID:244c08a5130c7e774ed4909b145d575a,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Created,Message:Created container: setup,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:14.163417078 +0000 UTC m=+4.058134499,LastTimestamp:2026-09-10 13:19:31.626544934 +0000 UTC m=+81.521262346,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:19:31.654396 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:31.654326 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal.18d3f82032d6445d\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal.18d3f82032d6445d openshift-machine-config-operator 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal,UID:244c08a5130c7e774ed4909b145d575a,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Started,Message:Started container setup,Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:14.171763805 +0000 UTC m=+4.066481222,LastTimestamp:2026-09-10 13:19:31.633576588 +0000 UTC m=+81.528293998,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:19:31.699163 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:31.699138 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal_244c08a5130c7e774ed4909b145d575a/setup/2.log" Sep 10 13:19:31.699278 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:31.699186 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal" event={"ID":"244c08a5130c7e774ed4909b145d575a","Type":"ContainerStarted","Data":"7a535ee8c5a571209ce138a868f916b873eb0df7e1475526b0bd073d7dc661de"} Sep 10 13:19:31.699321 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:31.699284 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:19:31.700069 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:31.700055 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:19:31.700155 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:31.700082 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:19:31.700155 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:31.700097 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:19:31.700283 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:31.700271 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-118.ec2.internal\" not found" node="ip-10-0-142-118.ec2.internal" Sep 10 13:19:32.431052 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:32.431017 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:33.302604 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:33.302566 2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s" Sep 10 13:19:33.421900 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:33.421874 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:33.565281 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:33.565207 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:19:33.566130 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:33.566108 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:19:33.566211 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:33.566145 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:19:33.566211 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:33.566155 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:19:33.566211 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:33.566182 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-118.ec2.internal" Sep 10 13:19:33.591959 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:33.591930 2560 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-142-118.ec2.internal" Sep 10 13:19:34.436530 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:34.436496 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:35.105705 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:35.105673 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 10 13:19:35.426684 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:35.426658 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:36.425539 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:36.425510 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:37.424071 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:37.424039 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:37.624584 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:37.624550 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" Sep 10 13:19:38.422384 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:38.422356 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:39.424717 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:39.424683 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:40.316133 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:40.316102 2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s" Sep 10 13:19:40.425185 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:40.425162 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:40.475883 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:40.475854 2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:40.539155 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:40.539122 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 10 13:19:40.592936 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:40.592844 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:19:40.593848 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:40.593825 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:19:40.593982 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:40.593857 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:19:40.593982 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:40.593871 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:19:40.593982 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:40.593902 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-118.ec2.internal" Sep 10 13:19:40.624212 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:40.624184 2560 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-142-118.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-142-118.ec2.internal" Sep 10 13:19:41.425249 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:41.425222 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:42.436878 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:42.436846 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:42.716242 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:42.716162 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal_244c08a5130c7e774ed4909b145d575a/setup/3.log" Sep 10 13:19:42.716552 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:42.716533 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal_244c08a5130c7e774ed4909b145d575a/setup/2.log" Sep 10 13:19:42.716632 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:42.716570 2560 generic.go:356] "Generic (PLEG): container finished" podID="244c08a5130c7e774ed4909b145d575a" containerID="7a535ee8c5a571209ce138a868f916b873eb0df7e1475526b0bd073d7dc661de" exitCode=1 Sep 10 13:19:42.716632 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:42.716598 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal" event={"ID":"244c08a5130c7e774ed4909b145d575a","Type":"ContainerDied","Data":"7a535ee8c5a571209ce138a868f916b873eb0df7e1475526b0bd073d7dc661de"} Sep 10 13:19:42.716725 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:42.716666 2560 scope.go:117] "RemoveContainer" containerID="d4a34d8eeb83ac3e673c74a5da9c728d37c2ee8e27353c6106a034f7f257f2a8" Sep 10 13:19:42.716774 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:42.716757 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:19:42.717830 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:42.717643 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:19:42.717830 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:42.717674 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:19:42.717830 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:42.717688 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:19:42.717991 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:42.717872 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-118.ec2.internal\" not found" node="ip-10-0-142-118.ec2.internal" Sep 10 13:19:42.717991 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:42.717921 2560 scope.go:117] "RemoveContainer" containerID="d4a34d8eeb83ac3e673c74a5da9c728d37c2ee8e27353c6106a034f7f257f2a8" Sep 10 13:19:42.725743 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:42.725712 2560 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal_openshift-machine-config-operator_244c08a5130c7e774ed4909b145d575a_2 in pod sandbox 26ad1edcb6fbc4feef3f6a5de973c50c88a37b9937dd4923725a7beb382e635e from index: no such id: 'd4a34d8eeb83ac3e673c74a5da9c728d37c2ee8e27353c6106a034f7f257f2a8'" containerID="d4a34d8eeb83ac3e673c74a5da9c728d37c2ee8e27353c6106a034f7f257f2a8" Sep 10 13:19:42.725816 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:42.725759 2560 kuberuntime_container.go:962] "Unhandled Error" err="failed to remove pod init container \"setup\": rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal_openshift-machine-config-operator_244c08a5130c7e774ed4909b145d575a_2 in pod sandbox 26ad1edcb6fbc4feef3f6a5de973c50c88a37b9937dd4923725a7beb382e635e from index: no such id: 'd4a34d8eeb83ac3e673c74a5da9c728d37c2ee8e27353c6106a034f7f257f2a8'; Skipping pod \"kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal_openshift-machine-config-operator(244c08a5130c7e774ed4909b145d575a)\"" logger="UnhandledError" Sep 10 13:19:42.725904 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:42.725887 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 40s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal_openshift-machine-config-operator(244c08a5130c7e774ed4909b145d575a)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal" podUID="244c08a5130c7e774ed4909b145d575a" Sep 10 13:19:42.738431 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:42.738344 2560 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal.18d3f825a87c71fb\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal.18d3f825a87c71fb openshift-machine-config-operator 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal,UID:244c08a5130c7e774ed4909b145d575a,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:BackOff,Message:Back-off restarting failed container setup in pod kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal_openshift-machine-config-operator(244c08a5130c7e774ed4909b145d575a),Source:EventSource{Component:kubelet,Host:ip-10-0-142-118.ec2.internal,},FirstTimestamp:2026-09-10 13:18:37.620425211 +0000 UTC m=+27.515142619,LastTimestamp:2026-09-10 13:19:42.725853929 +0000 UTC m=+92.620571338,Count:4,Type:Warning,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-118.ec2.internal,}" Sep 10 13:19:43.422140 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:43.422112 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:43.719942 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:43.719878 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal_244c08a5130c7e774ed4909b145d575a/setup/3.log" Sep 10 13:19:44.421334 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:44.421301 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:45.420849 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:45.420816 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:46.428045 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:46.428015 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-118.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 13:19:46.681668 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:46.681588 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 13:19:46.712743 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:46.712713 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 13:19:46.902384 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:46.902351 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-675z7" Sep 10 13:19:46.923897 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:46.923872 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-675z7" Sep 10 13:19:47.349461 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:47.349433 2560 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-142-118.ec2.internal\" not found" node="ip-10-0-142-118.ec2.internal" Sep 10 13:19:47.366555 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:47.366531 2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 10 13:19:47.443133 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:47.443109 2560 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-142-118.ec2.internal" not found Sep 10 13:19:47.468548 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:47.468527 2560 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-142-118.ec2.internal" not found Sep 10 13:19:47.550484 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:47.550458 2560 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-142-118.ec2.internal" not found Sep 10 13:19:47.625050 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:47.624997 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:19:47.625986 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:47.625970 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:19:47.626052 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:47.625998 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:19:47.626052 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:47.626008 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:19:47.626052 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:47.626038 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-118.ec2.internal" Sep 10 13:19:47.643011 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:47.642995 2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-118.ec2.internal" Sep 10 13:19:47.643062 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:47.643016 2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-118.ec2.internal\": node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:47.722165 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:47.722135 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:47.822643 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:47.822615 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:47.923551 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:47.923524 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:47.925679 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:47.925659 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 13:14:46 +0000 UTC" deadline="2028-05-26 14:12:35.18064637 +0000 UTC" Sep 10 13:19:47.925720 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:47.925681 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14976h52m47.254968602s" Sep 10 13:19:48.024170 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:48.024145 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:48.124599 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:48.124565 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:48.224936 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:48.224884 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:48.325276 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:48.325236 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:48.426070 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:48.426030 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:48.527084 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:48.527015 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:48.627859 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:48.627831 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:48.728237 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:48.728214 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:48.828827 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:48.828759 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:48.929021 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:48.928993 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:49.029517 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:49.029489 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:49.129911 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:49.129860 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:49.230874 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:49.230846 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:49.331534 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:49.331497 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:49.432039 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:49.432013 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:49.532673 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:49.532637 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:49.633262 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:49.633234 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:49.733792 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:49.733726 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:49.834234 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:49.834189 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:49.935047 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:49.935007 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:50.035565 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:50.035495 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:50.136013 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:50.135983 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:50.236526 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:50.236491 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:50.336985 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:50.336936 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:50.438024 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:50.437984 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:50.476471 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:50.476445 2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:50.538391 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:50.538360 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:50.639089 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:50.638998 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:50.739529 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:50.739501 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:50.839829 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:50.839790 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:50.939938 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:50.939901 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:51.040931 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:51.040894 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:51.141408 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:51.141366 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:51.242019 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:51.241946 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:51.342427 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:51.342388 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:51.443333 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:51.443299 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:51.544266 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:51.544209 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:51.644937 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:51.644904 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:51.745357 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:51.745331 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:51.845982 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:51.845909 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:51.946156 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:51.946127 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:52.046692 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:52.046657 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:52.147489 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:52.147440 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:52.248318 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:52.248291 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:52.348932 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:52.348894 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:52.450014 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:52.449993 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:52.550840 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:52.550804 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:52.651412 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:52.651374 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:52.751957 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:52.751892 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:52.852883 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:52.852848 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:52.953674 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:52.953634 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:53.054162 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:53.054095 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:53.154681 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:53.154642 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:53.255321 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:53.255282 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:53.355884 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:53.355834 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:53.456830 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:53.456787 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:53.557862 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:53.557820 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:53.658470 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:53.658391 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:53.759297 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:53.759257 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:53.860245 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:53.860209 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:53.961001 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:53.960970 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:54.061489 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:54.061453 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:54.162481 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:54.162444 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:54.263029 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:54.262970 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:54.363452 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:54.363424 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:54.464476 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:54.464448 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:54.564948 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:54.564899 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:54.665383 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:54.665359 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:54.766235 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:54.766201 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:54.867301 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:54.867239 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:54.967965 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:54.967937 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:55.068411 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:55.068370 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:55.168894 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:55.168865 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:55.269338 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:55.269313 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:55.370015 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:55.369985 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:55.471124 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:55.471050 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:55.571526 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:55.571495 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:55.672057 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:55.672028 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:55.772896 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:55.772832 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:55.873493 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:55.873448 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:55.974098 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:55.974056 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:56.074611 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:56.074551 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:56.175082 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:56.175038 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:56.275560 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:56.275525 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:56.376248 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:56.376185 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:56.477136 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:56.477102 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:56.578191 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:56.578158 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:56.678737 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:56.678709 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:56.779577 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:56.779535 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:56.880151 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:56.880112 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:56.981217 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:56.981151 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:57.081739 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:57.081706 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:57.182200 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:57.182159 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:57.282735 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:57.282670 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:57.383783 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:57.383744 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:57.483863 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:57.483827 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:57.556417 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:57.556355 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:19:57.557378 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:57.557358 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:19:57.557485 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:57.557390 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:19:57.557485 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:19:57.557417 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:19:57.557646 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:57.557631 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-118.ec2.internal\" not found" node="ip-10-0-142-118.ec2.internal" Sep 10 13:19:57.557811 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:57.557796 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 40s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal_openshift-machine-config-operator(244c08a5130c7e774ed4909b145d575a)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal" podUID="244c08a5130c7e774ed4909b145d575a" Sep 10 13:19:57.584238 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:57.584214 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:57.685059 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:57.685023 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:57.785492 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:57.785458 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:57.886186 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:57.886115 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:57.986837 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:57.986810 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:58.083294 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:58.083264 2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-118.ec2.internal\": node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:58.133851 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:58.133819 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:58.234220 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:58.234187 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:58.334776 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:58.334746 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:58.435155 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:58.435114 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:58.536064 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:58.536003 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:58.636884 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:58.636844 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:58.737361 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:58.737324 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:58.837861 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:58.837787 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:58.937943 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:58.937906 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:59.038434 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:59.038396 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:59.139420 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:59.139354 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:59.239828 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:59.239797 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:59.340267 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:59.340243 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:59.441227 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:59.441202 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:59.542078 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:59.542043 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:59.643173 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:59.643146 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:59.743643 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:59.743583 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:59.844535 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:59.844510 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:19:59.945252 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:19:59.945221 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:00.045851 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:00.045787 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:00.146283 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:00.146255 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:00.246844 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:00.246812 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:00.347305 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:00.347249 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:00.448224 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:00.448184 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:00.477550 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:00.477525 2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:00.548703 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:00.548662 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:00.649432 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:00.649334 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:00.749814 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:00.749792 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:00.850108 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:00.850074 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:00.950852 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:00.950831 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:01.051299 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:01.051271 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:01.151897 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:01.151871 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:01.252455 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:01.252379 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:01.352858 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:01.352825 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:01.452931 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:01.452886 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:01.553774 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:01.553700 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:01.654742 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:01.654705 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:01.755171 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:01.755127 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:01.855719 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:01.855654 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:01.956536 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:01.956498 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:02.057578 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:02.057539 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:02.158045 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:02.157990 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:02.259007 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:02.258977 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:02.359551 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:02.359515 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:02.459648 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:02.459604 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:02.560527 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:02.560490 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:02.660978 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:02.660951 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:02.761541 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:02.761474 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:02.861967 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:02.861931 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:02.962815 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:02.962772 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:03.063300 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:03.063232 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:03.163855 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:03.163824 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:03.264391 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:03.264351 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:03.365488 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:03.365417 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:03.466507 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:03.466473 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:03.567433 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:03.567388 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:03.668379 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:03.668353 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:03.768908 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:03.768879 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:03.869349 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:03.869317 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:03.970033 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:03.969968 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:04.070425 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:04.070383 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:04.170858 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:04.170807 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:04.271207 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:04.271117 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:04.371576 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:04.371537 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:04.472476 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:04.472437 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:04.572970 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:04.572893 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:04.673910 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:04.673877 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:04.774378 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:04.774332 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:04.875040 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:04.874973 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:04.975735 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:04.975697 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:05.076220 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:05.076179 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:05.176927 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:05.176901 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:05.277914 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:05.277883 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:05.378491 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:05.378459 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:05.478737 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:05.478672 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:05.579323 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:05.579289 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:05.679880 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:05.679844 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:05.780281 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:05.780198 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:05.881314 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:05.881274 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:05.982010 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:05.981986 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:06.082572 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:06.082509 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:06.182986 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:06.182953 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:06.283448 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:06.283415 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:06.384106 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:06.384038 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:06.484456 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:06.484430 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:06.585257 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:06.585220 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:06.685792 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:06.685764 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:06.786645 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:06.786607 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:06.887273 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:06.887231 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:06.988419 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:06.988332 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:07.088904 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:07.088871 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:07.189339 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:07.189299 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:07.289910 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:07.289841 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:07.389982 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:07.389958 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:07.490089 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:07.490053 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:07.590356 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:07.590289 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:07.691381 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:07.691336 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:07.792117 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:07.792081 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:07.892800 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:07.892725 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:07.993382 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:07.993338 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:08.093943 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:08.093907 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:08.194409 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:08.194371 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:08.294966 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:08.294938 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:08.314122 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:08.314088 2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-118.ec2.internal\": node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:08.395707 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:08.395677 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:08.496829 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:08.496753 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:08.597546 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:08.597506 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:08.698044 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:08.698005 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:08.798466 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:08.798378 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:08.899095 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:08.899058 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:08.999801 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:08.999763 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:09.100320 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:09.100253 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:09.200753 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:09.200723 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:09.301228 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:09.301190 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:09.402355 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:09.402288 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:09.503355 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:09.503317 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:09.604209 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:09.604175 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:09.705175 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:09.705146 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:09.805557 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:09.805524 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:09.906303 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:09.906269 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:10.006942 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:10.006851 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:10.107861 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:10.107827 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:10.208336 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:10.208290 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:10.308904 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:10.308832 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:10.409443 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:10.409417 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:10.478200 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:10.478163 2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:10.509677 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:10.509653 2560 kubelet_node_status.go:448] "Node not becoming ready in time after startup" Sep 10 13:20:12.557219 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:12.557185 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 13:20:12.558980 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:12.558964 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientMemory" Sep 10 13:20:12.559083 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:12.558992 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 13:20:12.559083 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:12.559002 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeHasSufficientPID" Sep 10 13:20:12.559233 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:12.559220 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-118.ec2.internal\" not found" node="ip-10-0-142-118.ec2.internal" Sep 10 13:20:12.559373 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:12.559357 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 40s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal_openshift-machine-config-operator(244c08a5130c7e774ed4909b145d575a)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal" podUID="244c08a5130c7e774ed4909b145d575a" Sep 10 13:20:15.493700 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:15.493665 2560 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 10 13:20:18.403519 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:18.403467 2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-118.ec2.internal\": node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:20.478610 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:20.478562 2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-118.ec2.internal\" not found" Sep 10 13:20:20.495020 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:20.494989 2560 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 10 13:20:23.752173 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:23.752143 2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 13:20:23.819168 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:23.819133 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal" Sep 10 13:20:23.837136 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:23.837117 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 13:20:23.838451 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:23.838436 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-118.ec2.internal" Sep 10 13:20:23.853118 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:23.853090 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 13:20:24.465999 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:24.465958 2560 apiserver.go:52] "Watching apiserver" Sep 10 13:20:24.469674 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:24.469658 2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 10 13:20:24.469826 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:24.469807 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-118.ec2.internal"] Sep 10 13:20:24.495428 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:24.495386 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 10 13:20:24.513050 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:24.513027 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 13:20:24.519566 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:24.519548 2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 10 13:20:24.549512 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:24.549491 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qktgr" Sep 10 13:20:24.778715 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:24.778632 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal_244c08a5130c7e774ed4909b145d575a/setup/3.log" Sep 10 13:20:24.778715 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:24.778687 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal" event={"ID":"244c08a5130c7e774ed4909b145d575a","Type":"ContainerStarted","Data":"fbd72b286ea27673367465d70a79fd7a6bed08977f147fb1ec1941740aa7a381"} Sep 10 13:20:25.496070 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:25.496031 2560 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 10 13:20:26.116334 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:26.116300 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qktgr" Sep 10 13:20:26.230732 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:26.230677 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-118.ec2.internal" podStartSLOduration=3.230646178 podStartE2EDuration="3.230646178s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:20:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:20:24.815667331 +0000 UTC m=+134.710384749" watchObservedRunningTime="2026-09-10 13:20:26.230646178 +0000 UTC m=+136.125363578" Sep 10 13:20:26.784161 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:26.784135 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal_244c08a5130c7e774ed4909b145d575a/setup/3.log" Sep 10 13:20:26.784314 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:26.784176 2560 generic.go:356] "Generic (PLEG): container finished" podID="244c08a5130c7e774ed4909b145d575a" containerID="fbd72b286ea27673367465d70a79fd7a6bed08977f147fb1ec1941740aa7a381" exitCode=0 Sep 10 13:20:26.784314 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:26.784206 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal" event={"ID":"244c08a5130c7e774ed4909b145d575a","Type":"ContainerDied","Data":"fbd72b286ea27673367465d70a79fd7a6bed08977f147fb1ec1941740aa7a381"} Sep 10 13:20:26.784314 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:26.784248 2560 scope.go:117] "RemoveContainer" containerID="7a535ee8c5a571209ce138a868f916b873eb0df7e1475526b0bd073d7dc661de" Sep 10 13:20:26.784566 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:26.784551 2560 scope.go:117] "RemoveContainer" containerID="7a535ee8c5a571209ce138a868f916b873eb0df7e1475526b0bd073d7dc661de" Sep 10 13:20:26.792793 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:26.792766 2560 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal_openshift-machine-config-operator_244c08a5130c7e774ed4909b145d575a_3 in pod sandbox 26ad1edcb6fbc4feef3f6a5de973c50c88a37b9937dd4923725a7beb382e635e from index: no such id: '7a535ee8c5a571209ce138a868f916b873eb0df7e1475526b0bd073d7dc661de'" containerID="7a535ee8c5a571209ce138a868f916b873eb0df7e1475526b0bd073d7dc661de" Sep 10 13:20:26.792869 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:26.792810 2560 kuberuntime_container.go:962] "Unhandled Error" err="failed to remove pod init container \"setup\": rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal_openshift-machine-config-operator_244c08a5130c7e774ed4909b145d575a_3 in pod sandbox 26ad1edcb6fbc4feef3f6a5de973c50c88a37b9937dd4923725a7beb382e635e from index: no such id: '7a535ee8c5a571209ce138a868f916b873eb0df7e1475526b0bd073d7dc661de'; Skipping pod \"kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal_openshift-machine-config-operator(244c08a5130c7e774ed4909b145d575a)\"" logger="UnhandledError" Sep 10 13:20:26.954038 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:26.954009 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-28xnt"] Sep 10 13:20:26.955375 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:26.955359 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-28xnt" Sep 10 13:20:26.959821 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:26.959804 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-dn7np"] Sep 10 13:20:26.961071 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:26.961050 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dn7np" Sep 10 13:20:26.962686 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:26.962670 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 10 13:20:26.965157 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:26.965142 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qmjpn\"" Sep 10 13:20:26.965325 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:26.965312 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 10 13:20:26.965874 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:26.965857 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 10 13:20:26.965933 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:26.965909 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 10 13:20:26.966885 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:26.966867 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 10 13:20:26.967035 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:26.967018 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7ztgj\"" Sep 10 13:20:26.980825 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:26.980807 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 10 13:20:27.039698 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.039668 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-hostroot\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.039803 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.039709 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vdz2\" (UniqueName: \"kubernetes.io/projected/6b168f32-20f0-4665-9eea-0ae159bd26fe-kube-api-access-4vdz2\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.039803 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.039728 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cc3dddf2-363d-4c1d-824f-aaca292a88f0-cnibin\") pod \"multus-additional-cni-plugins-28xnt\" (UID: \"cc3dddf2-363d-4c1d-824f-aaca292a88f0\") " pod="openshift-multus/multus-additional-cni-plugins-28xnt" Sep 10 13:20:27.039803 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.039743 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-system-cni-dir\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.039803 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.039763 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-host-var-lib-cni-bin\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.039803 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.039796 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-multus-conf-dir\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.039997 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.039821 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cc3dddf2-363d-4c1d-824f-aaca292a88f0-cni-binary-copy\") pod \"multus-additional-cni-plugins-28xnt\" (UID: \"cc3dddf2-363d-4c1d-824f-aaca292a88f0\") " pod="openshift-multus/multus-additional-cni-plugins-28xnt" Sep 10 13:20:27.039997 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.039839 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cc3dddf2-363d-4c1d-824f-aaca292a88f0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-28xnt\" (UID: \"cc3dddf2-363d-4c1d-824f-aaca292a88f0\") " pod="openshift-multus/multus-additional-cni-plugins-28xnt" Sep 10 13:20:27.039997 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.039854 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bvqf\" (UniqueName: \"kubernetes.io/projected/cc3dddf2-363d-4c1d-824f-aaca292a88f0-kube-api-access-9bvqf\") pod \"multus-additional-cni-plugins-28xnt\" (UID: \"cc3dddf2-363d-4c1d-824f-aaca292a88f0\") " pod="openshift-multus/multus-additional-cni-plugins-28xnt" Sep 10 13:20:27.039997 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.039872 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-cnibin\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.039997 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.039886 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-multus-socket-dir-parent\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.039997 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.039899 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-host-run-netns\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.039997 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.039912 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-host-var-lib-cni-multus\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.039997 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.039925 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-host-var-lib-kubelet\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.039997 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.039964 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc3dddf2-363d-4c1d-824f-aaca292a88f0-system-cni-dir\") pod \"multus-additional-cni-plugins-28xnt\" (UID: \"cc3dddf2-363d-4c1d-824f-aaca292a88f0\") " pod="openshift-multus/multus-additional-cni-plugins-28xnt" Sep 10 13:20:27.039997 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.039989 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-multus-cni-dir\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.040343 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.040004 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6b168f32-20f0-4665-9eea-0ae159bd26fe-multus-daemon-config\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.040343 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.040018 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-host-run-multus-certs\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.040343 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.040032 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-etc-kubernetes\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.040343 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.040048 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cc3dddf2-363d-4c1d-824f-aaca292a88f0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-28xnt\" (UID: \"cc3dddf2-363d-4c1d-824f-aaca292a88f0\") " pod="openshift-multus/multus-additional-cni-plugins-28xnt" Sep 10 13:20:27.040343 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.040063 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cc3dddf2-363d-4c1d-824f-aaca292a88f0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-28xnt\" (UID: \"cc3dddf2-363d-4c1d-824f-aaca292a88f0\") " pod="openshift-multus/multus-additional-cni-plugins-28xnt" Sep 10 13:20:27.040343 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.040109 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6b168f32-20f0-4665-9eea-0ae159bd26fe-cni-binary-copy\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.040343 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.040143 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-host-run-k8s-cni-cncf-io\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.118877 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.118806 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 13:15:26 +0000 UTC" deadline="2028-04-30 18:19:32.465775143 +0000 UTC" Sep 10 13:20:27.118877 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.118840 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14356h59m5.346937776s" Sep 10 13:20:27.141137 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.141109 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cc3dddf2-363d-4c1d-824f-aaca292a88f0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-28xnt\" (UID: \"cc3dddf2-363d-4c1d-824f-aaca292a88f0\") " pod="openshift-multus/multus-additional-cni-plugins-28xnt" Sep 10 13:20:27.141199 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.141139 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bvqf\" (UniqueName: \"kubernetes.io/projected/cc3dddf2-363d-4c1d-824f-aaca292a88f0-kube-api-access-9bvqf\") pod \"multus-additional-cni-plugins-28xnt\" (UID: \"cc3dddf2-363d-4c1d-824f-aaca292a88f0\") " pod="openshift-multus/multus-additional-cni-plugins-28xnt" Sep 10 13:20:27.141199 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.141155 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-cnibin\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.141199 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.141169 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-multus-socket-dir-parent\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.141199 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.141183 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-host-run-netns\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.141199 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.141196 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-host-var-lib-cni-multus\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.141453 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.141218 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-host-var-lib-kubelet\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.141453 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.141270 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-host-var-lib-kubelet\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.141453 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.141270 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-host-run-netns\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.141453 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.141280 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-host-var-lib-cni-multus\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.141453 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.141274 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-multus-socket-dir-parent\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.141453 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.141326 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-cnibin\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.141453 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.141328 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc3dddf2-363d-4c1d-824f-aaca292a88f0-system-cni-dir\") pod \"multus-additional-cni-plugins-28xnt\" (UID: \"cc3dddf2-363d-4c1d-824f-aaca292a88f0\") " pod="openshift-multus/multus-additional-cni-plugins-28xnt" Sep 10 13:20:27.141453 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.141358 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-multus-cni-dir\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.141453 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.141365 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc3dddf2-363d-4c1d-824f-aaca292a88f0-system-cni-dir\") pod \"multus-additional-cni-plugins-28xnt\" (UID: \"cc3dddf2-363d-4c1d-824f-aaca292a88f0\") " pod="openshift-multus/multus-additional-cni-plugins-28xnt" Sep 10 13:20:27.141453 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.141378 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6b168f32-20f0-4665-9eea-0ae159bd26fe-multus-daemon-config\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.141453 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.141394 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-host-run-multus-certs\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.141453 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.141387 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cc3dddf2-363d-4c1d-824f-aaca292a88f0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-28xnt\" (UID: \"cc3dddf2-363d-4c1d-824f-aaca292a88f0\") " pod="openshift-multus/multus-additional-cni-plugins-28xnt" Sep 10 13:20:27.141453 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.141428 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-etc-kubernetes\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.141453 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.141438 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-multus-cni-dir\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.142019 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.141467 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-etc-kubernetes\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.142019 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.141466 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-host-run-multus-certs\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.142019 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.141527 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cc3dddf2-363d-4c1d-824f-aaca292a88f0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-28xnt\" (UID: \"cc3dddf2-363d-4c1d-824f-aaca292a88f0\") " pod="openshift-multus/multus-additional-cni-plugins-28xnt" Sep 10 13:20:27.142019 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.141550 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cc3dddf2-363d-4c1d-824f-aaca292a88f0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-28xnt\" (UID: \"cc3dddf2-363d-4c1d-824f-aaca292a88f0\") " pod="openshift-multus/multus-additional-cni-plugins-28xnt" Sep 10 13:20:27.142019 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.141565 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6b168f32-20f0-4665-9eea-0ae159bd26fe-cni-binary-copy\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.142019 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.141586 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-host-run-k8s-cni-cncf-io\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.142019 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.141610 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-hostroot\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.142019 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.141657 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-hostroot\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.142019 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.141656 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-host-run-k8s-cni-cncf-io\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.142019 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.141677 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vdz2\" (UniqueName: \"kubernetes.io/projected/6b168f32-20f0-4665-9eea-0ae159bd26fe-kube-api-access-4vdz2\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.142019 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.141710 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cc3dddf2-363d-4c1d-824f-aaca292a88f0-cnibin\") pod \"multus-additional-cni-plugins-28xnt\" (UID: \"cc3dddf2-363d-4c1d-824f-aaca292a88f0\") " pod="openshift-multus/multus-additional-cni-plugins-28xnt" Sep 10 13:20:27.142019 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.141734 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-system-cni-dir\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.142019 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.141760 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-host-var-lib-cni-bin\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.142019 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.141783 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-multus-conf-dir\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.142019 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.141805 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cc3dddf2-363d-4c1d-824f-aaca292a88f0-cni-binary-copy\") pod \"multus-additional-cni-plugins-28xnt\" (UID: \"cc3dddf2-363d-4c1d-824f-aaca292a88f0\") " pod="openshift-multus/multus-additional-cni-plugins-28xnt" Sep 10 13:20:27.142019 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.141990 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-system-cni-dir\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.142618 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.142036 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cc3dddf2-363d-4c1d-824f-aaca292a88f0-cnibin\") pod \"multus-additional-cni-plugins-28xnt\" (UID: \"cc3dddf2-363d-4c1d-824f-aaca292a88f0\") " pod="openshift-multus/multus-additional-cni-plugins-28xnt" Sep 10 13:20:27.142618 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.142052 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6b168f32-20f0-4665-9eea-0ae159bd26fe-cni-binary-copy\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.142618 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.142075 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-host-var-lib-cni-bin\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.142618 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.142093 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6b168f32-20f0-4665-9eea-0ae159bd26fe-multus-conf-dir\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.142618 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.142102 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cc3dddf2-363d-4c1d-824f-aaca292a88f0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-28xnt\" (UID: \"cc3dddf2-363d-4c1d-824f-aaca292a88f0\") " pod="openshift-multus/multus-additional-cni-plugins-28xnt" Sep 10 13:20:27.142618 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.142052 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6b168f32-20f0-4665-9eea-0ae159bd26fe-multus-daemon-config\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.142618 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.142142 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cc3dddf2-363d-4c1d-824f-aaca292a88f0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-28xnt\" (UID: \"cc3dddf2-363d-4c1d-824f-aaca292a88f0\") " pod="openshift-multus/multus-additional-cni-plugins-28xnt" Sep 10 13:20:27.142618 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.142289 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cc3dddf2-363d-4c1d-824f-aaca292a88f0-cni-binary-copy\") pod \"multus-additional-cni-plugins-28xnt\" (UID: \"cc3dddf2-363d-4c1d-824f-aaca292a88f0\") " pod="openshift-multus/multus-additional-cni-plugins-28xnt" Sep 10 13:20:27.154853 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.154835 2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 13:20:27.175196 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:27.175179 2560 projected.go:291] Couldn't get configMap openshift-multus/kube-root-ca.crt: configmap "kube-root-ca.crt" not found Sep 10 13:20:27.175264 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:27.175200 2560 projected.go:196] Error preparing data for projected volume kube-api-access-4vdz2 for pod openshift-multus/multus-dn7np: configmap "kube-root-ca.crt" not found Sep 10 13:20:27.175304 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:27.175277 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b168f32-20f0-4665-9eea-0ae159bd26fe-kube-api-access-4vdz2 podName:6b168f32-20f0-4665-9eea-0ae159bd26fe nodeName:}" failed. No retries permitted until 2026-09-10 13:20:27.675262942 +0000 UTC m=+137.569980338 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4vdz2" (UniqueName: "kubernetes.io/projected/6b168f32-20f0-4665-9eea-0ae159bd26fe-kube-api-access-4vdz2") pod "multus-dn7np" (UID: "6b168f32-20f0-4665-9eea-0ae159bd26fe") : configmap "kube-root-ca.crt" not found Sep 10 13:20:27.177712 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:27.177698 2560 projected.go:291] Couldn't get configMap openshift-multus/kube-root-ca.crt: configmap "kube-root-ca.crt" not found Sep 10 13:20:27.177754 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:27.177718 2560 projected.go:196] Error preparing data for projected volume kube-api-access-9bvqf for pod openshift-multus/multus-additional-cni-plugins-28xnt: configmap "kube-root-ca.crt" not found Sep 10 13:20:27.177787 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:27.177762 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc3dddf2-363d-4c1d-824f-aaca292a88f0-kube-api-access-9bvqf podName:cc3dddf2-363d-4c1d-824f-aaca292a88f0 nodeName:}" failed. No retries permitted until 2026-09-10 13:20:27.677749148 +0000 UTC m=+137.572466545 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9bvqf" (UniqueName: "kubernetes.io/projected/cc3dddf2-363d-4c1d-824f-aaca292a88f0-kube-api-access-9bvqf") pod "multus-additional-cni-plugins-28xnt" (UID: "cc3dddf2-363d-4c1d-824f-aaca292a88f0") : configmap "kube-root-ca.crt" not found Sep 10 13:20:27.191303 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.191283 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-vrbmp"] Sep 10 13:20:27.193278 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.193264 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:20:27.193336 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:27.193321 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vrbmp" podUID="117705e7-b9e7-47ee-a474-d7f74afb23b6" Sep 10 13:20:27.225888 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.225863 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-g64cr"] Sep 10 13:20:27.227103 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.227089 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g64cr" Sep 10 13:20:27.227163 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:27.227148 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g64cr" podUID="3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597" Sep 10 13:20:27.242654 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.242630 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28v55\" (UniqueName: \"kubernetes.io/projected/3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597-kube-api-access-28v55\") pod \"network-check-target-g64cr\" (UID: \"3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597\") " pod="openshift-network-diagnostics/network-check-target-g64cr" Sep 10 13:20:27.242746 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.242668 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqwtj\" (UniqueName: \"kubernetes.io/projected/117705e7-b9e7-47ee-a474-d7f74afb23b6-kube-api-access-pqwtj\") pod \"network-metrics-daemon-vrbmp\" (UID: \"117705e7-b9e7-47ee-a474-d7f74afb23b6\") " pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:20:27.242746 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.242686 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/117705e7-b9e7-47ee-a474-d7f74afb23b6-metrics-certs\") pod \"network-metrics-daemon-vrbmp\" (UID: \"117705e7-b9e7-47ee-a474-d7f74afb23b6\") " pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:20:27.319689 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.319654 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-4sb6v"] Sep 10 13:20:27.322555 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.322524 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4sb6v" Sep 10 13:20:27.326973 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.326950 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 10 13:20:27.327085 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.326950 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 10 13:20:27.327937 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.327922 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 10 13:20:27.328022 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.328008 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-czbtf\"" Sep 10 13:20:27.343664 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.343638 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c7c8fe43-e33d-42a4-93fc-568959642979-host-slash\") pod \"iptables-alerter-4sb6v\" (UID: \"c7c8fe43-e33d-42a4-93fc-568959642979\") " pod="openshift-network-operator/iptables-alerter-4sb6v" Sep 10 13:20:27.343734 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.343677 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28v55\" (UniqueName: \"kubernetes.io/projected/3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597-kube-api-access-28v55\") pod \"network-check-target-g64cr\" (UID: \"3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597\") " pod="openshift-network-diagnostics/network-check-target-g64cr" Sep 10 13:20:27.343734 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.343693 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c7c8fe43-e33d-42a4-93fc-568959642979-iptables-alerter-script\") pod \"iptables-alerter-4sb6v\" (UID: \"c7c8fe43-e33d-42a4-93fc-568959642979\") " pod="openshift-network-operator/iptables-alerter-4sb6v" Sep 10 13:20:27.343807 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.343738 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqwtj\" (UniqueName: \"kubernetes.io/projected/117705e7-b9e7-47ee-a474-d7f74afb23b6-kube-api-access-pqwtj\") pod \"network-metrics-daemon-vrbmp\" (UID: \"117705e7-b9e7-47ee-a474-d7f74afb23b6\") " pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:20:27.343839 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.343815 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/117705e7-b9e7-47ee-a474-d7f74afb23b6-metrics-certs\") pod \"network-metrics-daemon-vrbmp\" (UID: \"117705e7-b9e7-47ee-a474-d7f74afb23b6\") " pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:20:27.343920 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.343847 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xr5sx\" (UniqueName: \"kubernetes.io/projected/c7c8fe43-e33d-42a4-93fc-568959642979-kube-api-access-xr5sx\") pod \"iptables-alerter-4sb6v\" (UID: \"c7c8fe43-e33d-42a4-93fc-568959642979\") " pod="openshift-network-operator/iptables-alerter-4sb6v" Sep 10 13:20:27.343920 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:27.343909 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 13:20:27.343978 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:27.343965 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/117705e7-b9e7-47ee-a474-d7f74afb23b6-metrics-certs podName:117705e7-b9e7-47ee-a474-d7f74afb23b6 nodeName:}" failed. No retries permitted until 2026-09-10 13:20:27.843951308 +0000 UTC m=+137.738668708 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/117705e7-b9e7-47ee-a474-d7f74afb23b6-metrics-certs") pod "network-metrics-daemon-vrbmp" (UID: "117705e7-b9e7-47ee-a474-d7f74afb23b6") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 13:20:27.361193 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:27.361175 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 13:20:27.361193 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:27.361194 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 13:20:27.361273 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:27.361204 2560 projected.go:196] Error preparing data for projected volume kube-api-access-28v55 for pod openshift-network-diagnostics/network-check-target-g64cr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 13:20:27.361273 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:27.361244 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597-kube-api-access-28v55 podName:3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597 nodeName:}" failed. No retries permitted until 2026-09-10 13:20:27.861231188 +0000 UTC m=+137.755948588 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-28v55" (UniqueName: "kubernetes.io/projected/3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597-kube-api-access-28v55") pod "network-check-target-g64cr" (UID: "3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 13:20:27.365451 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:27.365437 2560 projected.go:291] Couldn't get configMap openshift-multus/kube-root-ca.crt: configmap "kube-root-ca.crt" not found Sep 10 13:20:27.365491 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:27.365456 2560 projected.go:196] Error preparing data for projected volume kube-api-access-pqwtj for pod openshift-multus/network-metrics-daemon-vrbmp: configmap "kube-root-ca.crt" not found Sep 10 13:20:27.365522 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:27.365506 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/117705e7-b9e7-47ee-a474-d7f74afb23b6-kube-api-access-pqwtj podName:117705e7-b9e7-47ee-a474-d7f74afb23b6 nodeName:}" failed. No retries permitted until 2026-09-10 13:20:27.865493348 +0000 UTC m=+137.760210748 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-pqwtj" (UniqueName: "kubernetes.io/projected/117705e7-b9e7-47ee-a474-d7f74afb23b6-kube-api-access-pqwtj") pod "network-metrics-daemon-vrbmp" (UID: "117705e7-b9e7-47ee-a474-d7f74afb23b6") : configmap "kube-root-ca.crt" not found Sep 10 13:20:27.440864 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.440836 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-v2hld"] Sep 10 13:20:27.442186 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.442171 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:20:27.442245 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:27.442231 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v2hld" podUID="faa8bfdc-9c79-4c12-8ec9-3cd8639d0793" Sep 10 13:20:27.444673 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.444655 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c7c8fe43-e33d-42a4-93fc-568959642979-iptables-alerter-script\") pod \"iptables-alerter-4sb6v\" (UID: \"c7c8fe43-e33d-42a4-93fc-568959642979\") " pod="openshift-network-operator/iptables-alerter-4sb6v" Sep 10 13:20:27.444743 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.444704 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xr5sx\" (UniqueName: \"kubernetes.io/projected/c7c8fe43-e33d-42a4-93fc-568959642979-kube-api-access-xr5sx\") pod \"iptables-alerter-4sb6v\" (UID: \"c7c8fe43-e33d-42a4-93fc-568959642979\") " pod="openshift-network-operator/iptables-alerter-4sb6v" Sep 10 13:20:27.444743 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.444728 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c7c8fe43-e33d-42a4-93fc-568959642979-host-slash\") pod \"iptables-alerter-4sb6v\" (UID: \"c7c8fe43-e33d-42a4-93fc-568959642979\") " pod="openshift-network-operator/iptables-alerter-4sb6v" Sep 10 13:20:27.444808 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.444774 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c7c8fe43-e33d-42a4-93fc-568959642979-host-slash\") pod \"iptables-alerter-4sb6v\" (UID: \"c7c8fe43-e33d-42a4-93fc-568959642979\") " pod="openshift-network-operator/iptables-alerter-4sb6v" Sep 10 13:20:27.445187 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.445169 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c7c8fe43-e33d-42a4-93fc-568959642979-iptables-alerter-script\") pod \"iptables-alerter-4sb6v\" (UID: \"c7c8fe43-e33d-42a4-93fc-568959642979\") " pod="openshift-network-operator/iptables-alerter-4sb6v" Sep 10 13:20:27.448902 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.448886 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-t5w8t"] Sep 10 13:20:27.450611 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.450598 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.474614 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.474584 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 10 13:20:27.474614 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.474612 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 10 13:20:27.474888 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.474584 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 10 13:20:27.474888 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.474687 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 10 13:20:27.474888 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.474584 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 10 13:20:27.475098 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.475080 2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 10 13:20:27.475533 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.475510 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vx2vm\"" Sep 10 13:20:27.475652 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.475540 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 10 13:20:27.478684 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.478667 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xr5sx\" (UniqueName: \"kubernetes.io/projected/c7c8fe43-e33d-42a4-93fc-568959642979-kube-api-access-xr5sx\") pod \"iptables-alerter-4sb6v\" (UID: \"c7c8fe43-e33d-42a4-93fc-568959642979\") " pod="openshift-network-operator/iptables-alerter-4sb6v" Sep 10 13:20:27.499785 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.499757 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-2xql8"] Sep 10 13:20:27.501431 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.501414 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2xql8" Sep 10 13:20:27.508983 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.508966 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9hn4m\"" Sep 10 13:20:27.509588 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.509574 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 10 13:20:27.509870 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.509858 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 10 13:20:27.544994 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.544962 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0db61932-33cb-4fcd-84e6-79282c285bbf-ovn-node-metrics-cert\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.544994 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.544996 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0db61932-33cb-4fcd-84e6-79282c285bbf-ovnkube-config\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.545181 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.545020 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/df9a2932-82b3-4e64-9be2-9338269186b8-konnectivity-ca\") pod \"konnectivity-agent-2xql8\" (UID: \"df9a2932-82b3-4e64-9be2-9338269186b8\") " pod="kube-system/konnectivity-agent-2xql8" Sep 10 13:20:27.545181 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.545042 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/faa8bfdc-9c79-4c12-8ec9-3cd8639d0793-kubelet-config\") pod \"global-pull-secret-syncer-v2hld\" (UID: \"faa8bfdc-9c79-4c12-8ec9-3cd8639d0793\") " pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:20:27.545181 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.545079 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/df9a2932-82b3-4e64-9be2-9338269186b8-agent-certs\") pod \"konnectivity-agent-2xql8\" (UID: \"df9a2932-82b3-4e64-9be2-9338269186b8\") " pod="kube-system/konnectivity-agent-2xql8" Sep 10 13:20:27.545181 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.545106 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/faa8bfdc-9c79-4c12-8ec9-3cd8639d0793-dbus\") pod \"global-pull-secret-syncer-v2hld\" (UID: \"faa8bfdc-9c79-4c12-8ec9-3cd8639d0793\") " pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:20:27.545181 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.545122 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-run-netns\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.545181 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.545137 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-var-lib-openvswitch\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.545181 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.545152 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-etc-openvswitch\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.545181 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.545171 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-run-openvswitch\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.545181 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.545184 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-run-ovn\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.545548 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.545216 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-kubelet\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.545548 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.545241 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-systemd-units\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.545548 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.545263 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx7pm\" (UniqueName: \"kubernetes.io/projected/0db61932-33cb-4fcd-84e6-79282c285bbf-kube-api-access-lx7pm\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.545548 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.545290 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/faa8bfdc-9c79-4c12-8ec9-3cd8639d0793-original-pull-secret\") pod \"global-pull-secret-syncer-v2hld\" (UID: \"faa8bfdc-9c79-4c12-8ec9-3cd8639d0793\") " pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:20:27.545548 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.545309 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-run-systemd\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.545548 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.545326 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-log-socket\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.545548 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.545339 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0db61932-33cb-4fcd-84e6-79282c285bbf-ovnkube-script-lib\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.545548 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.545385 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.545548 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.545446 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-run-ovn-kubernetes\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.545548 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.545480 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-cni-bin\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.545548 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.545516 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-node-log\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.545548 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.545530 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-cni-netd\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.545548 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.545543 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0db61932-33cb-4fcd-84e6-79282c285bbf-env-overrides\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.545925 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.545559 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-slash\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.578155 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.578126 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpbx"] Sep 10 13:20:27.579742 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.579727 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpbx" Sep 10 13:20:27.586061 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.586033 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 10 13:20:27.586151 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.586068 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mzt68\"" Sep 10 13:20:27.586151 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.586103 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 10 13:20:27.586987 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.586974 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 10 13:20:27.630313 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.630286 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4sb6v" Sep 10 13:20:27.637468 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.637446 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-ps7jp"] Sep 10 13:20:27.638625 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:20:27.638603 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7c8fe43_e33d_42a4_93fc_568959642979.slice/crio-639612d7ece3f5723f46579faa95e3a5a1ff4eb9003556c817e8183ab04f6e2b WatchSource:0}: Error finding container 639612d7ece3f5723f46579faa95e3a5a1ff4eb9003556c817e8183ab04f6e2b: Status 404 returned error can't find the container with id 639612d7ece3f5723f46579faa95e3a5a1ff4eb9003556c817e8183ab04f6e2b Sep 10 13:20:27.639184 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.639167 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ps7jp" Sep 10 13:20:27.644318 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:27.644294 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"node-ca-ps7jp\" is forbidden: User \"system:node:ip-10-0-142-118.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-image-registry\": no relationship found between node 'ip-10-0-142-118.ec2.internal' and this object" podUID="611016d1-55f4-4537-b5cb-2bd50af1d7fb" pod="openshift-image-registry/node-ca-ps7jp" Sep 10 13:20:27.644449 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.644428 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 10 13:20:27.644514 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.644493 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 10 13:20:27.644554 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.644429 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 10 13:20:27.649791 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.646305 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a03086c6-fff1-4a62-b2b4-eb4d8266b1e3-etc-selinux\") pod \"aws-ebs-csi-driver-node-2xpbx\" (UID: \"a03086c6-fff1-4a62-b2b4-eb4d8266b1e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpbx" Sep 10 13:20:27.649791 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.646395 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0db61932-33cb-4fcd-84e6-79282c285bbf-ovn-node-metrics-cert\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.649791 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.646485 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0db61932-33cb-4fcd-84e6-79282c285bbf-ovnkube-config\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.649791 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.646521 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/df9a2932-82b3-4e64-9be2-9338269186b8-konnectivity-ca\") pod \"konnectivity-agent-2xql8\" (UID: \"df9a2932-82b3-4e64-9be2-9338269186b8\") " pod="kube-system/konnectivity-agent-2xql8" Sep 10 13:20:27.649791 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.646578 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/faa8bfdc-9c79-4c12-8ec9-3cd8639d0793-kubelet-config\") pod \"global-pull-secret-syncer-v2hld\" (UID: \"faa8bfdc-9c79-4c12-8ec9-3cd8639d0793\") " pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:20:27.649791 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.646611 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdh4m\" (UniqueName: \"kubernetes.io/projected/a03086c6-fff1-4a62-b2b4-eb4d8266b1e3-kube-api-access-vdh4m\") pod \"aws-ebs-csi-driver-node-2xpbx\" (UID: \"a03086c6-fff1-4a62-b2b4-eb4d8266b1e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpbx" Sep 10 13:20:27.649791 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.646648 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/df9a2932-82b3-4e64-9be2-9338269186b8-agent-certs\") pod \"konnectivity-agent-2xql8\" (UID: \"df9a2932-82b3-4e64-9be2-9338269186b8\") " pod="kube-system/konnectivity-agent-2xql8" Sep 10 13:20:27.649791 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.646681 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/faa8bfdc-9c79-4c12-8ec9-3cd8639d0793-dbus\") pod \"global-pull-secret-syncer-v2hld\" (UID: \"faa8bfdc-9c79-4c12-8ec9-3cd8639d0793\") " pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:20:27.649791 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.646716 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-run-netns\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.649791 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.646749 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-var-lib-openvswitch\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.649791 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.646775 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-etc-openvswitch\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.649791 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.646815 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-run-openvswitch\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.649791 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.646851 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-run-ovn\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.649791 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.646887 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a03086c6-fff1-4a62-b2b4-eb4d8266b1e3-device-dir\") pod \"aws-ebs-csi-driver-node-2xpbx\" (UID: \"a03086c6-fff1-4a62-b2b4-eb4d8266b1e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpbx" Sep 10 13:20:27.649791 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.646921 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-kubelet\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.649791 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.646951 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-systemd-units\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.649791 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.646980 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx7pm\" (UniqueName: \"kubernetes.io/projected/0db61932-33cb-4fcd-84e6-79282c285bbf-kube-api-access-lx7pm\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.650489 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.647018 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/faa8bfdc-9c79-4c12-8ec9-3cd8639d0793-original-pull-secret\") pod \"global-pull-secret-syncer-v2hld\" (UID: \"faa8bfdc-9c79-4c12-8ec9-3cd8639d0793\") " pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:20:27.650489 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.647051 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-run-systemd\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.650489 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.647091 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-log-socket\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.650489 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.647120 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0db61932-33cb-4fcd-84e6-79282c285bbf-ovnkube-script-lib\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.650489 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.647188 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.650489 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.647250 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-run-ovn-kubernetes\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.650489 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.647283 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-cni-bin\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.650489 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.647310 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-node-log\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.650489 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.647341 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-cni-netd\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.650489 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.647369 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0db61932-33cb-4fcd-84e6-79282c285bbf-env-overrides\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.650489 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.647421 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a03086c6-fff1-4a62-b2b4-eb4d8266b1e3-socket-dir\") pod \"aws-ebs-csi-driver-node-2xpbx\" (UID: \"a03086c6-fff1-4a62-b2b4-eb4d8266b1e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpbx" Sep 10 13:20:27.650489 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.647455 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a03086c6-fff1-4a62-b2b4-eb4d8266b1e3-registration-dir\") pod \"aws-ebs-csi-driver-node-2xpbx\" (UID: \"a03086c6-fff1-4a62-b2b4-eb4d8266b1e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpbx" Sep 10 13:20:27.650489 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.647550 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-slash\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.650489 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.647584 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a03086c6-fff1-4a62-b2b4-eb4d8266b1e3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2xpbx\" (UID: \"a03086c6-fff1-4a62-b2b4-eb4d8266b1e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpbx" Sep 10 13:20:27.650489 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.647620 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a03086c6-fff1-4a62-b2b4-eb4d8266b1e3-sys-fs\") pod \"aws-ebs-csi-driver-node-2xpbx\" (UID: \"a03086c6-fff1-4a62-b2b4-eb4d8266b1e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpbx" Sep 10 13:20:27.650489 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.647722 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-var-lib-openvswitch\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.650489 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.647748 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.651054 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.647808 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-etc-openvswitch\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.651054 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.647820 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-cni-bin\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.651054 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.647876 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-run-ovn-kubernetes\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.651054 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.647914 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-slash\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.651054 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.647955 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-cni-netd\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.651054 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.648010 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-run-netns\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.651054 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.648025 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-run-openvswitch\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.651054 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.648077 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-run-ovn\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.651054 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.648078 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/faa8bfdc-9c79-4c12-8ec9-3cd8639d0793-dbus\") pod \"global-pull-secret-syncer-v2hld\" (UID: \"faa8bfdc-9c79-4c12-8ec9-3cd8639d0793\") " pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:20:27.651054 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.648134 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-kubelet\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.651054 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.648156 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/faa8bfdc-9c79-4c12-8ec9-3cd8639d0793-kubelet-config\") pod \"global-pull-secret-syncer-v2hld\" (UID: \"faa8bfdc-9c79-4c12-8ec9-3cd8639d0793\") " pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:20:27.651054 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.648207 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-systemd-units\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.651054 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.648218 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-log-socket\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.651054 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.648258 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-run-systemd\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.651054 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:27.648352 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 13:20:27.651054 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:27.648426 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faa8bfdc-9c79-4c12-8ec9-3cd8639d0793-original-pull-secret podName:faa8bfdc-9c79-4c12-8ec9-3cd8639d0793 nodeName:}" failed. No retries permitted until 2026-09-10 13:20:28.148394437 +0000 UTC m=+138.043111844 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/faa8bfdc-9c79-4c12-8ec9-3cd8639d0793-original-pull-secret") pod "global-pull-secret-syncer-v2hld" (UID: "faa8bfdc-9c79-4c12-8ec9-3cd8639d0793") : object "kube-system"/"original-pull-secret" not registered Sep 10 13:20:27.651054 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.648481 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-node-log\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.651552 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.648678 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/df9a2932-82b3-4e64-9be2-9338269186b8-konnectivity-ca\") pod \"konnectivity-agent-2xql8\" (UID: \"df9a2932-82b3-4e64-9be2-9338269186b8\") " pod="kube-system/konnectivity-agent-2xql8" Sep 10 13:20:27.651552 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.648819 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0db61932-33cb-4fcd-84e6-79282c285bbf-env-overrides\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.651552 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.648926 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0db61932-33cb-4fcd-84e6-79282c285bbf-ovnkube-script-lib\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.651552 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.649199 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0db61932-33cb-4fcd-84e6-79282c285bbf-ovnkube-config\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.651552 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.650537 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0db61932-33cb-4fcd-84e6-79282c285bbf-ovn-node-metrics-cert\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.652322 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.652300 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/df9a2932-82b3-4e64-9be2-9338269186b8-agent-certs\") pod \"konnectivity-agent-2xql8\" (UID: \"df9a2932-82b3-4e64-9be2-9338269186b8\") " pod="kube-system/konnectivity-agent-2xql8" Sep 10 13:20:27.653274 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.653261 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rg9tv\"" Sep 10 13:20:27.680443 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.680392 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-6lgkb"] Sep 10 13:20:27.682460 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.682444 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.685515 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.685493 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx7pm\" (UniqueName: \"kubernetes.io/projected/0db61932-33cb-4fcd-84e6-79282c285bbf-kube-api-access-lx7pm\") pod \"ovnkube-node-t5w8t\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.687561 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.687523 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 10 13:20:27.687657 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.687633 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-lmg57\"" Sep 10 13:20:27.689869 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.689852 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 10 13:20:27.748383 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.748307 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a03086c6-fff1-4a62-b2b4-eb4d8266b1e3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2xpbx\" (UID: \"a03086c6-fff1-4a62-b2b4-eb4d8266b1e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpbx" Sep 10 13:20:27.748383 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.748339 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a03086c6-fff1-4a62-b2b4-eb4d8266b1e3-sys-fs\") pod \"aws-ebs-csi-driver-node-2xpbx\" (UID: \"a03086c6-fff1-4a62-b2b4-eb4d8266b1e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpbx" Sep 10 13:20:27.748383 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.748359 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-etc-sysctl-d\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.748383 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.748382 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a03086c6-fff1-4a62-b2b4-eb4d8266b1e3-etc-selinux\") pod \"aws-ebs-csi-driver-node-2xpbx\" (UID: \"a03086c6-fff1-4a62-b2b4-eb4d8266b1e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpbx" Sep 10 13:20:27.748626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.748417 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-etc-tuned\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.748626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.748448 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bvqf\" (UniqueName: \"kubernetes.io/projected/cc3dddf2-363d-4c1d-824f-aaca292a88f0-kube-api-access-9bvqf\") pod \"multus-additional-cni-plugins-28xnt\" (UID: \"cc3dddf2-363d-4c1d-824f-aaca292a88f0\") " pod="openshift-multus/multus-additional-cni-plugins-28xnt" Sep 10 13:20:27.748626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.748453 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a03086c6-fff1-4a62-b2b4-eb4d8266b1e3-sys-fs\") pod \"aws-ebs-csi-driver-node-2xpbx\" (UID: \"a03086c6-fff1-4a62-b2b4-eb4d8266b1e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpbx" Sep 10 13:20:27.748626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.748450 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a03086c6-fff1-4a62-b2b4-eb4d8266b1e3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2xpbx\" (UID: \"a03086c6-fff1-4a62-b2b4-eb4d8266b1e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpbx" Sep 10 13:20:27.748626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.748503 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdh4m\" (UniqueName: \"kubernetes.io/projected/a03086c6-fff1-4a62-b2b4-eb4d8266b1e3-kube-api-access-vdh4m\") pod \"aws-ebs-csi-driver-node-2xpbx\" (UID: \"a03086c6-fff1-4a62-b2b4-eb4d8266b1e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpbx" Sep 10 13:20:27.748626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.748521 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtxb6\" (UniqueName: \"kubernetes.io/projected/611016d1-55f4-4537-b5cb-2bd50af1d7fb-kube-api-access-dtxb6\") pod \"node-ca-ps7jp\" (UID: \"611016d1-55f4-4537-b5cb-2bd50af1d7fb\") " pod="openshift-image-registry/node-ca-ps7jp" Sep 10 13:20:27.748626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.748522 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a03086c6-fff1-4a62-b2b4-eb4d8266b1e3-etc-selinux\") pod \"aws-ebs-csi-driver-node-2xpbx\" (UID: \"a03086c6-fff1-4a62-b2b4-eb4d8266b1e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpbx" Sep 10 13:20:27.748626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.748536 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-tmp\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.748626 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:27.748608 2560 projected.go:291] Couldn't get configMap openshift-multus/kube-root-ca.crt: configmap "kube-root-ca.crt" not found Sep 10 13:20:27.748949 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:27.748634 2560 projected.go:196] Error preparing data for projected volume kube-api-access-9bvqf for pod openshift-multus/multus-additional-cni-plugins-28xnt: configmap "kube-root-ca.crt" not found Sep 10 13:20:27.748949 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.748644 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a03086c6-fff1-4a62-b2b4-eb4d8266b1e3-device-dir\") pod \"aws-ebs-csi-driver-node-2xpbx\" (UID: \"a03086c6-fff1-4a62-b2b4-eb4d8266b1e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpbx" Sep 10 13:20:27.748949 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.748659 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/611016d1-55f4-4537-b5cb-2bd50af1d7fb-host\") pod \"node-ca-ps7jp\" (UID: \"611016d1-55f4-4537-b5cb-2bd50af1d7fb\") " pod="openshift-image-registry/node-ca-ps7jp" Sep 10 13:20:27.748949 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:27.748687 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc3dddf2-363d-4c1d-824f-aaca292a88f0-kube-api-access-9bvqf podName:cc3dddf2-363d-4c1d-824f-aaca292a88f0 nodeName:}" failed. No retries permitted until 2026-09-10 13:20:28.748667885 +0000 UTC m=+138.643385292 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9bvqf" (UniqueName: "kubernetes.io/projected/cc3dddf2-363d-4c1d-824f-aaca292a88f0-kube-api-access-9bvqf") pod "multus-additional-cni-plugins-28xnt" (UID: "cc3dddf2-363d-4c1d-824f-aaca292a88f0") : configmap "kube-root-ca.crt" not found Sep 10 13:20:27.748949 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.748696 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a03086c6-fff1-4a62-b2b4-eb4d8266b1e3-device-dir\") pod \"aws-ebs-csi-driver-node-2xpbx\" (UID: \"a03086c6-fff1-4a62-b2b4-eb4d8266b1e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpbx" Sep 10 13:20:27.748949 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.748707 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-etc-modprobe-d\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.748949 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.748740 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-run\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.748949 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.748789 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-etc-kubernetes\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.748949 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.748805 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-lib-modules\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.748949 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.748827 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-host\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.748949 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.748856 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/611016d1-55f4-4537-b5cb-2bd50af1d7fb-serviceca\") pod \"node-ca-ps7jp\" (UID: \"611016d1-55f4-4537-b5cb-2bd50af1d7fb\") " pod="openshift-image-registry/node-ca-ps7jp" Sep 10 13:20:27.748949 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.748875 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-sys\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.748949 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.748912 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vdz2\" (UniqueName: \"kubernetes.io/projected/6b168f32-20f0-4665-9eea-0ae159bd26fe-kube-api-access-4vdz2\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:27.748949 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.748941 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-etc-sysconfig\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.749348 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.749006 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-etc-systemd\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.749348 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:27.749023 2560 projected.go:291] Couldn't get configMap openshift-multus/kube-root-ca.crt: configmap "kube-root-ca.crt" not found Sep 10 13:20:27.749348 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.749033 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-var-lib-kubelet\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.749348 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:27.749040 2560 projected.go:196] Error preparing data for projected volume kube-api-access-4vdz2 for pod openshift-multus/multus-dn7np: configmap "kube-root-ca.crt" not found Sep 10 13:20:27.749348 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:27.749077 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b168f32-20f0-4665-9eea-0ae159bd26fe-kube-api-access-4vdz2 podName:6b168f32-20f0-4665-9eea-0ae159bd26fe nodeName:}" failed. No retries permitted until 2026-09-10 13:20:28.749068177 +0000 UTC m=+138.643785572 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4vdz2" (UniqueName: "kubernetes.io/projected/6b168f32-20f0-4665-9eea-0ae159bd26fe-kube-api-access-4vdz2") pod "multus-dn7np" (UID: "6b168f32-20f0-4665-9eea-0ae159bd26fe") : configmap "kube-root-ca.crt" not found Sep 10 13:20:27.749348 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.749103 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7n5zb\" (UniqueName: \"kubernetes.io/projected/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-kube-api-access-7n5zb\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.749348 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.749127 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a03086c6-fff1-4a62-b2b4-eb4d8266b1e3-socket-dir\") pod \"aws-ebs-csi-driver-node-2xpbx\" (UID: \"a03086c6-fff1-4a62-b2b4-eb4d8266b1e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpbx" Sep 10 13:20:27.749348 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.749143 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a03086c6-fff1-4a62-b2b4-eb4d8266b1e3-registration-dir\") pod \"aws-ebs-csi-driver-node-2xpbx\" (UID: \"a03086c6-fff1-4a62-b2b4-eb4d8266b1e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpbx" Sep 10 13:20:27.749348 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.749158 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-etc-sysctl-conf\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.749348 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.749246 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a03086c6-fff1-4a62-b2b4-eb4d8266b1e3-registration-dir\") pod \"aws-ebs-csi-driver-node-2xpbx\" (UID: \"a03086c6-fff1-4a62-b2b4-eb4d8266b1e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpbx" Sep 10 13:20:27.749348 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.749243 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a03086c6-fff1-4a62-b2b4-eb4d8266b1e3-socket-dir\") pod \"aws-ebs-csi-driver-node-2xpbx\" (UID: \"a03086c6-fff1-4a62-b2b4-eb4d8266b1e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpbx" Sep 10 13:20:27.758014 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.757996 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:27.765115 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:20:27.765093 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0db61932_33cb_4fcd_84e6_79282c285bbf.slice/crio-14fd596d470db1c9ae38b85eb33b88527b6572430bbab1903f02a876334e614e WatchSource:0}: Error finding container 14fd596d470db1c9ae38b85eb33b88527b6572430bbab1903f02a876334e614e: Status 404 returned error can't find the container with id 14fd596d470db1c9ae38b85eb33b88527b6572430bbab1903f02a876334e614e Sep 10 13:20:27.775698 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.775673 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdh4m\" (UniqueName: \"kubernetes.io/projected/a03086c6-fff1-4a62-b2b4-eb4d8266b1e3-kube-api-access-vdh4m\") pod \"aws-ebs-csi-driver-node-2xpbx\" (UID: \"a03086c6-fff1-4a62-b2b4-eb4d8266b1e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpbx" Sep 10 13:20:27.786510 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.786480 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4sb6v" event={"ID":"c7c8fe43-e33d-42a4-93fc-568959642979","Type":"ContainerStarted","Data":"639612d7ece3f5723f46579faa95e3a5a1ff4eb9003556c817e8183ab04f6e2b"} Sep 10 13:20:27.788129 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.788112 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal" event={"ID":"244c08a5130c7e774ed4909b145d575a","Type":"ContainerStarted","Data":"68e970f0ee44690829d1c9f16b594724af84a176de1a908131ec469cc90b900f"} Sep 10 13:20:27.789095 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.789076 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" event={"ID":"0db61932-33cb-4fcd-84e6-79282c285bbf","Type":"ContainerStarted","Data":"14fd596d470db1c9ae38b85eb33b88527b6572430bbab1903f02a876334e614e"} Sep 10 13:20:27.808376 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.808352 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2xql8" Sep 10 13:20:27.815385 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:20:27.815363 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf9a2932_82b3_4e64_9be2_9338269186b8.slice/crio-b804eeeeec469be241be33ebb5abc6d593b744499619978298dbc7c756e48d1c WatchSource:0}: Error finding container b804eeeeec469be241be33ebb5abc6d593b744499619978298dbc7c756e48d1c: Status 404 returned error can't find the container with id b804eeeeec469be241be33ebb5abc6d593b744499619978298dbc7c756e48d1c Sep 10 13:20:27.849646 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.849621 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-etc-kubernetes\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.849646 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.849648 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-lib-modules\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.849820 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.849662 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-host\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.849820 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.849676 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/611016d1-55f4-4537-b5cb-2bd50af1d7fb-serviceca\") pod \"node-ca-ps7jp\" (UID: \"611016d1-55f4-4537-b5cb-2bd50af1d7fb\") " pod="openshift-image-registry/node-ca-ps7jp" Sep 10 13:20:27.849820 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.849689 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-sys\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.849820 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.849715 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-etc-sysconfig\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.849820 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.849734 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/117705e7-b9e7-47ee-a474-d7f74afb23b6-metrics-certs\") pod \"network-metrics-daemon-vrbmp\" (UID: \"117705e7-b9e7-47ee-a474-d7f74afb23b6\") " pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:20:27.849820 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.849757 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-etc-systemd\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.849820 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.849754 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-etc-kubernetes\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.849820 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.849797 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-sys\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.849820 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.849802 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-etc-sysconfig\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.849820 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.849797 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-lib-modules\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.850221 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.849805 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-var-lib-kubelet\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.850221 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.849836 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-etc-systemd\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.850221 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.849855 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-host\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.850221 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.849859 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-var-lib-kubelet\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.850221 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.849861 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7n5zb\" (UniqueName: \"kubernetes.io/projected/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-kube-api-access-7n5zb\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.850221 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:27.849854 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 13:20:27.850221 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.849886 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-etc-sysctl-conf\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.850221 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.849922 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-etc-sysctl-d\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.850221 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.849939 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-etc-tuned\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.850221 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.849974 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtxb6\" (UniqueName: \"kubernetes.io/projected/611016d1-55f4-4537-b5cb-2bd50af1d7fb-kube-api-access-dtxb6\") pod \"node-ca-ps7jp\" (UID: \"611016d1-55f4-4537-b5cb-2bd50af1d7fb\") " pod="openshift-image-registry/node-ca-ps7jp" Sep 10 13:20:27.850221 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.849995 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-tmp\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.850221 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.850005 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-etc-sysctl-conf\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.850221 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.850020 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/611016d1-55f4-4537-b5cb-2bd50af1d7fb-host\") pod \"node-ca-ps7jp\" (UID: \"611016d1-55f4-4537-b5cb-2bd50af1d7fb\") " pod="openshift-image-registry/node-ca-ps7jp" Sep 10 13:20:27.850221 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.850024 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-etc-sysctl-d\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.850221 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:27.850047 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/117705e7-b9e7-47ee-a474-d7f74afb23b6-metrics-certs podName:117705e7-b9e7-47ee-a474-d7f74afb23b6 nodeName:}" failed. No retries permitted until 2026-09-10 13:20:28.850026056 +0000 UTC m=+138.744743459 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/117705e7-b9e7-47ee-a474-d7f74afb23b6-metrics-certs") pod "network-metrics-daemon-vrbmp" (UID: "117705e7-b9e7-47ee-a474-d7f74afb23b6") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 13:20:27.850221 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.850066 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/611016d1-55f4-4537-b5cb-2bd50af1d7fb-host\") pod \"node-ca-ps7jp\" (UID: \"611016d1-55f4-4537-b5cb-2bd50af1d7fb\") " pod="openshift-image-registry/node-ca-ps7jp" Sep 10 13:20:27.850221 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.850084 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-etc-modprobe-d\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.850221 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.850112 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-run\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.850869 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.850174 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-run\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.850869 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.850183 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-etc-modprobe-d\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.850869 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.850718 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/611016d1-55f4-4537-b5cb-2bd50af1d7fb-serviceca\") pod \"node-ca-ps7jp\" (UID: \"611016d1-55f4-4537-b5cb-2bd50af1d7fb\") " pod="openshift-image-registry/node-ca-ps7jp" Sep 10 13:20:27.852163 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.852145 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-tmp\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.852294 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.852279 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-etc-tuned\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.884224 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.884202 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtxb6\" (UniqueName: \"kubernetes.io/projected/611016d1-55f4-4537-b5cb-2bd50af1d7fb-kube-api-access-dtxb6\") pod \"node-ca-ps7jp\" (UID: \"611016d1-55f4-4537-b5cb-2bd50af1d7fb\") " pod="openshift-image-registry/node-ca-ps7jp" Sep 10 13:20:27.887138 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.887123 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpbx" Sep 10 13:20:27.894557 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:20:27.894531 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda03086c6_fff1_4a62_b2b4_eb4d8266b1e3.slice/crio-758562232d99cbb1239adbeb8f4f6e70ac52c281c5095a763164a36b72cfec8d WatchSource:0}: Error finding container 758562232d99cbb1239adbeb8f4f6e70ac52c281c5095a763164a36b72cfec8d: Status 404 returned error can't find the container with id 758562232d99cbb1239adbeb8f4f6e70ac52c281c5095a763164a36b72cfec8d Sep 10 13:20:27.928236 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.928207 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7n5zb\" (UniqueName: \"kubernetes.io/projected/4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c-kube-api-access-7n5zb\") pod \"tuned-6lgkb\" (UID: \"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c\") " pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:27.951132 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.951097 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqwtj\" (UniqueName: \"kubernetes.io/projected/117705e7-b9e7-47ee-a474-d7f74afb23b6-kube-api-access-pqwtj\") pod \"network-metrics-daemon-vrbmp\" (UID: \"117705e7-b9e7-47ee-a474-d7f74afb23b6\") " pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:20:27.951211 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.951163 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28v55\" (UniqueName: \"kubernetes.io/projected/3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597-kube-api-access-28v55\") pod \"network-check-target-g64cr\" (UID: \"3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597\") " pod="openshift-network-diagnostics/network-check-target-g64cr" Sep 10 13:20:27.951263 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:27.951246 2560 projected.go:291] Couldn't get configMap openshift-multus/kube-root-ca.crt: configmap "kube-root-ca.crt" not found Sep 10 13:20:27.951302 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:27.951271 2560 projected.go:196] Error preparing data for projected volume kube-api-access-pqwtj for pod openshift-multus/network-metrics-daemon-vrbmp: configmap "kube-root-ca.crt" not found Sep 10 13:20:27.951302 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:27.951274 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 13:20:27.951302 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:27.951288 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 13:20:27.951302 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:27.951297 2560 projected.go:196] Error preparing data for projected volume kube-api-access-28v55 for pod openshift-network-diagnostics/network-check-target-g64cr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 13:20:27.951428 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:27.951315 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/117705e7-b9e7-47ee-a474-d7f74afb23b6-kube-api-access-pqwtj podName:117705e7-b9e7-47ee-a474-d7f74afb23b6 nodeName:}" failed. No retries permitted until 2026-09-10 13:20:28.951301469 +0000 UTC m=+138.846018871 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-pqwtj" (UniqueName: "kubernetes.io/projected/117705e7-b9e7-47ee-a474-d7f74afb23b6-kube-api-access-pqwtj") pod "network-metrics-daemon-vrbmp" (UID: "117705e7-b9e7-47ee-a474-d7f74afb23b6") : configmap "kube-root-ca.crt" not found Sep 10 13:20:27.951428 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:27.951330 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597-kube-api-access-28v55 podName:3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597 nodeName:}" failed. No retries permitted until 2026-09-10 13:20:28.951320139 +0000 UTC m=+138.846037535 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-28v55" (UniqueName: "kubernetes.io/projected/3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597-kube-api-access-28v55") pod "network-check-target-g64cr" (UID: "3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 13:20:27.957119 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.957092 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ps7jp" Sep 10 13:20:27.964518 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:20:27.964498 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod611016d1_55f4_4537_b5cb_2bd50af1d7fb.slice/crio-058f72afad33fbfd7c203c9f02023793a7d359a8435aa40542ea828f68c4288d WatchSource:0}: Error finding container 058f72afad33fbfd7c203c9f02023793a7d359a8435aa40542ea828f68c4288d: Status 404 returned error can't find the container with id 058f72afad33fbfd7c203c9f02023793a7d359a8435aa40542ea828f68c4288d Sep 10 13:20:27.993544 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:27.993521 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" Sep 10 13:20:28.000043 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:20:27.999995 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bfed6c3_60c9_4e2f_bed1_7614b7ff8f8c.slice/crio-cff6a11e03a0e251e3a9656f9e8c69c8e466cfd865c0c802b7adf4c6e5f45e19 WatchSource:0}: Error finding container cff6a11e03a0e251e3a9656f9e8c69c8e466cfd865c0c802b7adf4c6e5f45e19: Status 404 returned error can't find the container with id cff6a11e03a0e251e3a9656f9e8c69c8e466cfd865c0c802b7adf4c6e5f45e19 Sep 10 13:20:28.118991 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:28.118934 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 13:15:26 +0000 UTC" deadline="2028-06-14 10:15:11.227959706 +0000 UTC" Sep 10 13:20:28.118991 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:28.118983 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15428h54m43.108979207s" Sep 10 13:20:28.152375 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:28.152341 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/faa8bfdc-9c79-4c12-8ec9-3cd8639d0793-original-pull-secret\") pod \"global-pull-secret-syncer-v2hld\" (UID: \"faa8bfdc-9c79-4c12-8ec9-3cd8639d0793\") " pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:20:28.152512 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:28.152499 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 13:20:28.152570 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:28.152560 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faa8bfdc-9c79-4c12-8ec9-3cd8639d0793-original-pull-secret podName:faa8bfdc-9c79-4c12-8ec9-3cd8639d0793 nodeName:}" failed. No retries permitted until 2026-09-10 13:20:29.152545848 +0000 UTC m=+139.047263244 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/faa8bfdc-9c79-4c12-8ec9-3cd8639d0793-original-pull-secret") pod "global-pull-secret-syncer-v2hld" (UID: "faa8bfdc-9c79-4c12-8ec9-3cd8639d0793") : object "kube-system"/"original-pull-secret" not registered Sep 10 13:20:28.557129 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:28.557059 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g64cr" Sep 10 13:20:28.557313 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:28.557059 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:20:28.557313 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:28.557193 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g64cr" podUID="3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597" Sep 10 13:20:28.557313 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:28.557205 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:20:28.557495 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:28.557324 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vrbmp" podUID="117705e7-b9e7-47ee-a474-d7f74afb23b6" Sep 10 13:20:28.557495 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:28.557426 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v2hld" podUID="faa8bfdc-9c79-4c12-8ec9-3cd8639d0793" Sep 10 13:20:28.757094 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:28.757047 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vdz2\" (UniqueName: \"kubernetes.io/projected/6b168f32-20f0-4665-9eea-0ae159bd26fe-kube-api-access-4vdz2\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:28.757276 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:28.757167 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bvqf\" (UniqueName: \"kubernetes.io/projected/cc3dddf2-363d-4c1d-824f-aaca292a88f0-kube-api-access-9bvqf\") pod \"multus-additional-cni-plugins-28xnt\" (UID: \"cc3dddf2-363d-4c1d-824f-aaca292a88f0\") " pod="openshift-multus/multus-additional-cni-plugins-28xnt" Sep 10 13:20:28.760370 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:28.760343 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bvqf\" (UniqueName: \"kubernetes.io/projected/cc3dddf2-363d-4c1d-824f-aaca292a88f0-kube-api-access-9bvqf\") pod \"multus-additional-cni-plugins-28xnt\" (UID: \"cc3dddf2-363d-4c1d-824f-aaca292a88f0\") " pod="openshift-multus/multus-additional-cni-plugins-28xnt" Sep 10 13:20:28.760749 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:28.760729 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vdz2\" (UniqueName: \"kubernetes.io/projected/6b168f32-20f0-4665-9eea-0ae159bd26fe-kube-api-access-4vdz2\") pod \"multus-dn7np\" (UID: \"6b168f32-20f0-4665-9eea-0ae159bd26fe\") " pod="openshift-multus/multus-dn7np" Sep 10 13:20:28.763202 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:28.763002 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-28xnt" Sep 10 13:20:28.769350 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:28.769222 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dn7np" Sep 10 13:20:28.772369 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:20:28.772327 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc3dddf2_363d_4c1d_824f_aaca292a88f0.slice/crio-dd65a31dd1b8c6f57d04fc69daf633f53b454938951d84b11602d4238f09b7d6 WatchSource:0}: Error finding container dd65a31dd1b8c6f57d04fc69daf633f53b454938951d84b11602d4238f09b7d6: Status 404 returned error can't find the container with id dd65a31dd1b8c6f57d04fc69daf633f53b454938951d84b11602d4238f09b7d6 Sep 10 13:20:28.778882 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:20:28.778716 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b168f32_20f0_4665_9eea_0ae159bd26fe.slice/crio-1a480ea929cd38302615d77f2048f14928b44d3ae6821507b6c60fe38202b724 WatchSource:0}: Error finding container 1a480ea929cd38302615d77f2048f14928b44d3ae6821507b6c60fe38202b724: Status 404 returned error can't find the container with id 1a480ea929cd38302615d77f2048f14928b44d3ae6821507b6c60fe38202b724 Sep 10 13:20:28.792748 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:28.792714 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-28xnt" event={"ID":"cc3dddf2-363d-4c1d-824f-aaca292a88f0","Type":"ContainerStarted","Data":"dd65a31dd1b8c6f57d04fc69daf633f53b454938951d84b11602d4238f09b7d6"} Sep 10 13:20:28.794471 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:28.794382 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" event={"ID":"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c","Type":"ContainerStarted","Data":"cff6a11e03a0e251e3a9656f9e8c69c8e466cfd865c0c802b7adf4c6e5f45e19"} Sep 10 13:20:28.795837 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:28.795720 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ps7jp" event={"ID":"611016d1-55f4-4537-b5cb-2bd50af1d7fb","Type":"ContainerStarted","Data":"058f72afad33fbfd7c203c9f02023793a7d359a8435aa40542ea828f68c4288d"} Sep 10 13:20:28.797354 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:28.797278 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpbx" event={"ID":"a03086c6-fff1-4a62-b2b4-eb4d8266b1e3","Type":"ContainerStarted","Data":"758562232d99cbb1239adbeb8f4f6e70ac52c281c5095a763164a36b72cfec8d"} Sep 10 13:20:28.799146 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:28.799124 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2xql8" event={"ID":"df9a2932-82b3-4e64-9be2-9338269186b8","Type":"ContainerStarted","Data":"b804eeeeec469be241be33ebb5abc6d593b744499619978298dbc7c756e48d1c"} Sep 10 13:20:28.801315 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:28.801294 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dn7np" event={"ID":"6b168f32-20f0-4665-9eea-0ae159bd26fe","Type":"ContainerStarted","Data":"1a480ea929cd38302615d77f2048f14928b44d3ae6821507b6c60fe38202b724"} Sep 10 13:20:28.858470 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:28.858379 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/117705e7-b9e7-47ee-a474-d7f74afb23b6-metrics-certs\") pod \"network-metrics-daemon-vrbmp\" (UID: \"117705e7-b9e7-47ee-a474-d7f74afb23b6\") " pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:20:28.858628 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:28.858555 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 13:20:28.858628 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:28.858619 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/117705e7-b9e7-47ee-a474-d7f74afb23b6-metrics-certs podName:117705e7-b9e7-47ee-a474-d7f74afb23b6 nodeName:}" failed. No retries permitted until 2026-09-10 13:20:30.858601682 +0000 UTC m=+140.753319081 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/117705e7-b9e7-47ee-a474-d7f74afb23b6-metrics-certs") pod "network-metrics-daemon-vrbmp" (UID: "117705e7-b9e7-47ee-a474-d7f74afb23b6") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 13:20:28.959619 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:28.959574 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqwtj\" (UniqueName: \"kubernetes.io/projected/117705e7-b9e7-47ee-a474-d7f74afb23b6-kube-api-access-pqwtj\") pod \"network-metrics-daemon-vrbmp\" (UID: \"117705e7-b9e7-47ee-a474-d7f74afb23b6\") " pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:20:28.959790 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:28.959659 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28v55\" (UniqueName: \"kubernetes.io/projected/3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597-kube-api-access-28v55\") pod \"network-check-target-g64cr\" (UID: \"3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597\") " pod="openshift-network-diagnostics/network-check-target-g64cr" Sep 10 13:20:28.959858 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:28.959801 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 13:20:28.959858 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:28.959818 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 13:20:28.959858 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:28.959831 2560 projected.go:196] Error preparing data for projected volume kube-api-access-28v55 for pod openshift-network-diagnostics/network-check-target-g64cr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 13:20:28.960009 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:28.959886 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597-kube-api-access-28v55 podName:3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597 nodeName:}" failed. No retries permitted until 2026-09-10 13:20:30.959869188 +0000 UTC m=+140.854586586 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-28v55" (UniqueName: "kubernetes.io/projected/3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597-kube-api-access-28v55") pod "network-check-target-g64cr" (UID: "3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 13:20:28.971530 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:28.971477 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqwtj\" (UniqueName: \"kubernetes.io/projected/117705e7-b9e7-47ee-a474-d7f74afb23b6-kube-api-access-pqwtj\") pod \"network-metrics-daemon-vrbmp\" (UID: \"117705e7-b9e7-47ee-a474-d7f74afb23b6\") " pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:20:29.161819 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:29.161199 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/faa8bfdc-9c79-4c12-8ec9-3cd8639d0793-original-pull-secret\") pod \"global-pull-secret-syncer-v2hld\" (UID: \"faa8bfdc-9c79-4c12-8ec9-3cd8639d0793\") " pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:20:29.161819 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:29.161357 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 13:20:29.161819 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:29.161437 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faa8bfdc-9c79-4c12-8ec9-3cd8639d0793-original-pull-secret podName:faa8bfdc-9c79-4c12-8ec9-3cd8639d0793 nodeName:}" failed. No retries permitted until 2026-09-10 13:20:31.161418689 +0000 UTC m=+141.056136106 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/faa8bfdc-9c79-4c12-8ec9-3cd8639d0793-original-pull-secret") pod "global-pull-secret-syncer-v2hld" (UID: "faa8bfdc-9c79-4c12-8ec9-3cd8639d0793") : object "kube-system"/"original-pull-secret" not registered Sep 10 13:20:30.497513 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:30.496337 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 13:20:30.497513 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:30.497088 2560 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 10 13:20:30.558837 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:30.558127 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:20:30.558837 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:30.558250 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vrbmp" podUID="117705e7-b9e7-47ee-a474-d7f74afb23b6" Sep 10 13:20:30.558837 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:30.558600 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:20:30.558837 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:30.558686 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v2hld" podUID="faa8bfdc-9c79-4c12-8ec9-3cd8639d0793" Sep 10 13:20:30.558837 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:30.558738 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g64cr" Sep 10 13:20:30.558837 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:30.558801 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g64cr" podUID="3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597" Sep 10 13:20:30.582022 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:30.581014 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-118.ec2.internal" podStartSLOduration=7.580994316 podStartE2EDuration="7.580994316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:20:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:20:27.831268901 +0000 UTC m=+137.725986319" watchObservedRunningTime="2026-09-10 13:20:30.580994316 +0000 UTC m=+140.475711736" Sep 10 13:20:30.875890 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:30.875809 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/117705e7-b9e7-47ee-a474-d7f74afb23b6-metrics-certs\") pod \"network-metrics-daemon-vrbmp\" (UID: \"117705e7-b9e7-47ee-a474-d7f74afb23b6\") " pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:20:30.876041 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:30.875988 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 13:20:30.876096 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:30.876064 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/117705e7-b9e7-47ee-a474-d7f74afb23b6-metrics-certs podName:117705e7-b9e7-47ee-a474-d7f74afb23b6 nodeName:}" failed. No retries permitted until 2026-09-10 13:20:34.876043628 +0000 UTC m=+144.770761025 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/117705e7-b9e7-47ee-a474-d7f74afb23b6-metrics-certs") pod "network-metrics-daemon-vrbmp" (UID: "117705e7-b9e7-47ee-a474-d7f74afb23b6") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 13:20:30.977111 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:30.977071 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28v55\" (UniqueName: \"kubernetes.io/projected/3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597-kube-api-access-28v55\") pod \"network-check-target-g64cr\" (UID: \"3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597\") " pod="openshift-network-diagnostics/network-check-target-g64cr" Sep 10 13:20:30.977306 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:30.977272 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 13:20:30.977306 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:30.977299 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 13:20:30.977469 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:30.977315 2560 projected.go:196] Error preparing data for projected volume kube-api-access-28v55 for pod openshift-network-diagnostics/network-check-target-g64cr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 13:20:30.977469 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:30.977375 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597-kube-api-access-28v55 podName:3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597 nodeName:}" failed. No retries permitted until 2026-09-10 13:20:34.977354999 +0000 UTC m=+144.872072411 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-28v55" (UniqueName: "kubernetes.io/projected/3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597-kube-api-access-28v55") pod "network-check-target-g64cr" (UID: "3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 13:20:31.178944 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:31.178907 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/faa8bfdc-9c79-4c12-8ec9-3cd8639d0793-original-pull-secret\") pod \"global-pull-secret-syncer-v2hld\" (UID: \"faa8bfdc-9c79-4c12-8ec9-3cd8639d0793\") " pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:20:31.179129 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:31.179081 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 13:20:31.179186 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:31.179166 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faa8bfdc-9c79-4c12-8ec9-3cd8639d0793-original-pull-secret podName:faa8bfdc-9c79-4c12-8ec9-3cd8639d0793 nodeName:}" failed. No retries permitted until 2026-09-10 13:20:35.179145418 +0000 UTC m=+145.073862815 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/faa8bfdc-9c79-4c12-8ec9-3cd8639d0793-original-pull-secret") pod "global-pull-secret-syncer-v2hld" (UID: "faa8bfdc-9c79-4c12-8ec9-3cd8639d0793") : object "kube-system"/"original-pull-secret" not registered Sep 10 13:20:32.556954 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:32.556923 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:20:32.557421 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:32.556937 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:20:32.557421 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:32.556939 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g64cr" Sep 10 13:20:32.557421 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:32.557071 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vrbmp" podUID="117705e7-b9e7-47ee-a474-d7f74afb23b6" Sep 10 13:20:32.557421 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:32.557174 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g64cr" podUID="3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597" Sep 10 13:20:32.557421 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:32.557257 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v2hld" podUID="faa8bfdc-9c79-4c12-8ec9-3cd8639d0793" Sep 10 13:20:33.587816 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:33.587781 2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 13:20:34.557221 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:34.557192 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:20:34.557221 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:34.557226 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g64cr" Sep 10 13:20:34.557538 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:34.557192 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:20:34.557538 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:34.557334 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vrbmp" podUID="117705e7-b9e7-47ee-a474-d7f74afb23b6" Sep 10 13:20:34.557538 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:34.557425 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g64cr" podUID="3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597" Sep 10 13:20:34.557538 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:34.557507 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v2hld" podUID="faa8bfdc-9c79-4c12-8ec9-3cd8639d0793" Sep 10 13:20:34.903933 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:34.903848 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/117705e7-b9e7-47ee-a474-d7f74afb23b6-metrics-certs\") pod \"network-metrics-daemon-vrbmp\" (UID: \"117705e7-b9e7-47ee-a474-d7f74afb23b6\") " pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:20:34.904392 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:34.904006 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 13:20:34.904392 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:34.904079 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/117705e7-b9e7-47ee-a474-d7f74afb23b6-metrics-certs podName:117705e7-b9e7-47ee-a474-d7f74afb23b6 nodeName:}" failed. No retries permitted until 2026-09-10 13:20:42.904057364 +0000 UTC m=+152.798774775 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/117705e7-b9e7-47ee-a474-d7f74afb23b6-metrics-certs") pod "network-metrics-daemon-vrbmp" (UID: "117705e7-b9e7-47ee-a474-d7f74afb23b6") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 13:20:35.005153 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:35.005111 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28v55\" (UniqueName: \"kubernetes.io/projected/3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597-kube-api-access-28v55\") pod \"network-check-target-g64cr\" (UID: \"3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597\") " pod="openshift-network-diagnostics/network-check-target-g64cr" Sep 10 13:20:35.005324 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:35.005292 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 13:20:35.005324 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:35.005317 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 13:20:35.005446 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:35.005330 2560 projected.go:196] Error preparing data for projected volume kube-api-access-28v55 for pod openshift-network-diagnostics/network-check-target-g64cr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 13:20:35.005446 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:35.005388 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597-kube-api-access-28v55 podName:3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597 nodeName:}" failed. No retries permitted until 2026-09-10 13:20:43.005369007 +0000 UTC m=+152.900086408 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-28v55" (UniqueName: "kubernetes.io/projected/3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597-kube-api-access-28v55") pod "network-check-target-g64cr" (UID: "3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 13:20:35.207597 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:35.206945 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/faa8bfdc-9c79-4c12-8ec9-3cd8639d0793-original-pull-secret\") pod \"global-pull-secret-syncer-v2hld\" (UID: \"faa8bfdc-9c79-4c12-8ec9-3cd8639d0793\") " pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:20:35.207597 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:35.207157 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 13:20:35.207597 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:35.207226 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faa8bfdc-9c79-4c12-8ec9-3cd8639d0793-original-pull-secret podName:faa8bfdc-9c79-4c12-8ec9-3cd8639d0793 nodeName:}" failed. No retries permitted until 2026-09-10 13:20:43.207205364 +0000 UTC m=+153.101922777 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/faa8bfdc-9c79-4c12-8ec9-3cd8639d0793-original-pull-secret") pod "global-pull-secret-syncer-v2hld" (UID: "faa8bfdc-9c79-4c12-8ec9-3cd8639d0793") : object "kube-system"/"original-pull-secret" not registered Sep 10 13:20:35.498816 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:35.498714 2560 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 10 13:20:36.557314 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:36.556645 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g64cr" Sep 10 13:20:36.557314 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:36.556778 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g64cr" podUID="3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597" Sep 10 13:20:36.557314 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:36.557159 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:20:36.557314 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:36.557261 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vrbmp" podUID="117705e7-b9e7-47ee-a474-d7f74afb23b6" Sep 10 13:20:36.557957 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:36.557425 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:20:36.557957 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:36.557535 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v2hld" podUID="faa8bfdc-9c79-4c12-8ec9-3cd8639d0793" Sep 10 13:20:38.557228 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:38.557195 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:20:38.557679 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:38.557195 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:20:38.557679 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:38.557315 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v2hld" podUID="faa8bfdc-9c79-4c12-8ec9-3cd8639d0793" Sep 10 13:20:38.557679 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:38.557195 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g64cr" Sep 10 13:20:38.557679 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:38.557386 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vrbmp" podUID="117705e7-b9e7-47ee-a474-d7f74afb23b6" Sep 10 13:20:38.560290 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:38.557905 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g64cr" podUID="3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597" Sep 10 13:20:38.725169 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:38.725135 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-t5w8t"] Sep 10 13:20:40.499180 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:40.499142 2560 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 10 13:20:40.557773 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:40.557695 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:20:40.557916 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:40.557796 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:20:40.557916 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:40.557834 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g64cr" Sep 10 13:20:40.557916 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:40.557836 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vrbmp" podUID="117705e7-b9e7-47ee-a474-d7f74afb23b6" Sep 10 13:20:40.557916 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:40.557905 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g64cr" podUID="3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597" Sep 10 13:20:40.558103 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:40.557979 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v2hld" podUID="faa8bfdc-9c79-4c12-8ec9-3cd8639d0793" Sep 10 13:20:42.556722 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:42.556681 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g64cr" Sep 10 13:20:42.557183 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:42.556690 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:20:42.557183 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:42.556819 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g64cr" podUID="3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597" Sep 10 13:20:42.557183 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:42.556904 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v2hld" podUID="faa8bfdc-9c79-4c12-8ec9-3cd8639d0793" Sep 10 13:20:42.557183 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:42.556690 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:20:42.557183 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:42.557008 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vrbmp" podUID="117705e7-b9e7-47ee-a474-d7f74afb23b6" Sep 10 13:20:42.963479 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:42.963442 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/117705e7-b9e7-47ee-a474-d7f74afb23b6-metrics-certs\") pod \"network-metrics-daemon-vrbmp\" (UID: \"117705e7-b9e7-47ee-a474-d7f74afb23b6\") " pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:20:42.963658 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:42.963633 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 13:20:42.963728 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:42.963720 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/117705e7-b9e7-47ee-a474-d7f74afb23b6-metrics-certs podName:117705e7-b9e7-47ee-a474-d7f74afb23b6 nodeName:}" failed. No retries permitted until 2026-09-10 13:20:58.963697878 +0000 UTC m=+168.858415279 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/117705e7-b9e7-47ee-a474-d7f74afb23b6-metrics-certs") pod "network-metrics-daemon-vrbmp" (UID: "117705e7-b9e7-47ee-a474-d7f74afb23b6") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 13:20:43.064365 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:43.064322 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28v55\" (UniqueName: \"kubernetes.io/projected/3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597-kube-api-access-28v55\") pod \"network-check-target-g64cr\" (UID: \"3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597\") " pod="openshift-network-diagnostics/network-check-target-g64cr" Sep 10 13:20:43.064553 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:43.064530 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 13:20:43.064600 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:43.064559 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 13:20:43.064600 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:43.064570 2560 projected.go:196] Error preparing data for projected volume kube-api-access-28v55 for pod openshift-network-diagnostics/network-check-target-g64cr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 13:20:43.064683 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:43.064619 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597-kube-api-access-28v55 podName:3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597 nodeName:}" failed. No retries permitted until 2026-09-10 13:20:59.064604523 +0000 UTC m=+168.959321919 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-28v55" (UniqueName: "kubernetes.io/projected/3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597-kube-api-access-28v55") pod "network-check-target-g64cr" (UID: "3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 13:20:43.267046 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:43.266950 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/faa8bfdc-9c79-4c12-8ec9-3cd8639d0793-original-pull-secret\") pod \"global-pull-secret-syncer-v2hld\" (UID: \"faa8bfdc-9c79-4c12-8ec9-3cd8639d0793\") " pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:20:43.267192 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:43.267125 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 13:20:43.267233 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:43.267194 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faa8bfdc-9c79-4c12-8ec9-3cd8639d0793-original-pull-secret podName:faa8bfdc-9c79-4c12-8ec9-3cd8639d0793 nodeName:}" failed. No retries permitted until 2026-09-10 13:20:59.267175727 +0000 UTC m=+169.161893127 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/faa8bfdc-9c79-4c12-8ec9-3cd8639d0793-original-pull-secret") pod "global-pull-secret-syncer-v2hld" (UID: "faa8bfdc-9c79-4c12-8ec9-3cd8639d0793") : object "kube-system"/"original-pull-secret" not registered Sep 10 13:20:44.557837 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:44.557541 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g64cr" Sep 10 13:20:44.558509 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:44.557837 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g64cr" podUID="3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597" Sep 10 13:20:44.558509 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:44.557911 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:20:44.558509 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:44.558000 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vrbmp" podUID="117705e7-b9e7-47ee-a474-d7f74afb23b6" Sep 10 13:20:44.558509 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:44.558053 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:20:44.558509 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:44.558122 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v2hld" podUID="faa8bfdc-9c79-4c12-8ec9-3cd8639d0793" Sep 10 13:20:44.836851 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:44.836770 2560 generic.go:356] "Generic (PLEG): container finished" podID="cc3dddf2-363d-4c1d-824f-aaca292a88f0" containerID="4d8fa0e57b4906ff068aaedcbd2adee6c4f25ceb405ba7c523caf210d71de096" exitCode=0 Sep 10 13:20:44.836986 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:44.836856 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-28xnt" event={"ID":"cc3dddf2-363d-4c1d-824f-aaca292a88f0","Type":"ContainerDied","Data":"4d8fa0e57b4906ff068aaedcbd2adee6c4f25ceb405ba7c523caf210d71de096"} Sep 10 13:20:44.838549 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:44.838523 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" event={"ID":"4bfed6c3-60c9-4e2f-bed1-7614b7ff8f8c","Type":"ContainerStarted","Data":"dcda49dad1141515992379fbe12af8abc1ac55f306e595b1a0cbd06ed91a7efc"} Sep 10 13:20:44.840137 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:44.840111 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ps7jp" event={"ID":"611016d1-55f4-4537-b5cb-2bd50af1d7fb","Type":"ContainerStarted","Data":"3e0b29f32b21202bc80c813c1269abf835dddcc018de4ff986bab9f4ddb6b200"} Sep 10 13:20:44.841766 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:44.841742 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpbx" event={"ID":"a03086c6-fff1-4a62-b2b4-eb4d8266b1e3","Type":"ContainerStarted","Data":"cc75f8820b7b839bd99808392597b99c1aa2d407ec017819ec609b5a2112b48b"} Sep 10 13:20:44.843180 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:44.843155 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2xql8" event={"ID":"df9a2932-82b3-4e64-9be2-9338269186b8","Type":"ContainerStarted","Data":"8bbc18da378f2ef5b8bfdf3377b1956a055c23a5b6206e535cf17b339ff1257f"} Sep 10 13:20:44.846534 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:44.846516 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" event={"ID":"0db61932-33cb-4fcd-84e6-79282c285bbf","Type":"ContainerStarted","Data":"f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3"} Sep 10 13:20:44.846607 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:44.846576 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" event={"ID":"0db61932-33cb-4fcd-84e6-79282c285bbf","Type":"ContainerStarted","Data":"ac691669ea7a5766986b91d76178c6feae5635a7049a196d165a08a5a92c1c8a"} Sep 10 13:20:44.846661 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:44.846627 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" event={"ID":"0db61932-33cb-4fcd-84e6-79282c285bbf","Type":"ContainerStarted","Data":"3b987da2b6daccdf548c60335fb33085cf2d3d58027b5562894c64b117b0f610"} Sep 10 13:20:44.846698 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:44.846676 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" event={"ID":"0db61932-33cb-4fcd-84e6-79282c285bbf","Type":"ContainerStarted","Data":"1d6a5cbf688a0e4d0d9bd3c2538e5ebd4d830b54a409fd038a41a5f0ab92fecf"} Sep 10 13:20:44.846737 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:44.846723 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" event={"ID":"0db61932-33cb-4fcd-84e6-79282c285bbf","Type":"ContainerStarted","Data":"d612f51b0b053f9abd930117864ad6f708f5fad939202d5c0974cf73a1b2b2c9"} Sep 10 13:20:44.846778 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:44.846759 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" event={"ID":"0db61932-33cb-4fcd-84e6-79282c285bbf","Type":"ContainerStarted","Data":"3fd215ba1d9dd060316b5b3ad36f8a035bc01f4603e770d22b85639c5eb8aed9"} Sep 10 13:20:44.847926 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:44.847908 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dn7np" event={"ID":"6b168f32-20f0-4665-9eea-0ae159bd26fe","Type":"ContainerStarted","Data":"f80db502728adcc9cd0dcf2e7d7b6ffa51a45a4ead69060c58155d1b9626df74"} Sep 10 13:20:44.875663 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:44.875623 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-dn7np" podStartSLOduration=3.219486718 podStartE2EDuration="18.875608442s" totalImagesPullingTime="15.656121724s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:20:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:20:44.875597483 +0000 UTC m=+154.770314901" watchObservedRunningTime="2026-09-10 13:20:44.875608442 +0000 UTC m=+154.770325858" Sep 10 13:20:44.933998 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:44.933937 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-2xql8" podStartSLOduration=1.374412561 podStartE2EDuration="17.933919469s" totalImagesPullingTime="16.559506908s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:20:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:20:44.901921227 +0000 UTC m=+154.796638646" watchObservedRunningTime="2026-09-10 13:20:44.933919469 +0000 UTC m=+154.828636877" Sep 10 13:20:44.966445 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:44.966379 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-6lgkb" podStartSLOduration=1.588244638 podStartE2EDuration="17.966364905s" totalImagesPullingTime="16.378120267s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:20:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:20:44.966189678 +0000 UTC m=+154.860907096" watchObservedRunningTime="2026-09-10 13:20:44.966364905 +0000 UTC m=+154.861082324" Sep 10 13:20:44.966673 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:44.966655 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ps7jp" podStartSLOduration=1.615998472 podStartE2EDuration="17.966649398s" totalImagesPullingTime="16.350650926s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:20:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:20:44.941044472 +0000 UTC m=+154.835761891" watchObservedRunningTime="2026-09-10 13:20:44.966649398 +0000 UTC m=+154.861366816" Sep 10 13:20:45.500161 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:45.500016 2560 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 10 13:20:45.508321 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:45.508300 2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 10 13:20:45.554932 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:45.554837 2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T13:20:45.508317123Z","UUID":"c40eb864-0ff6-4e69-8647-9afeb0c64f59","Handler":null,"Name":"","Endpoint":""} Sep 10 13:20:45.556159 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:45.556142 2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 10 13:20:45.556273 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:45.556169 2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 10 13:20:45.851807 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:45.851766 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpbx" event={"ID":"a03086c6-fff1-4a62-b2b4-eb4d8266b1e3","Type":"ContainerStarted","Data":"3ab2f2cc09bd21d63eb3173630b30f4473847000aed8f34b0caa630adeaa364b"} Sep 10 13:20:45.853076 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:45.853052 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4sb6v" event={"ID":"c7c8fe43-e33d-42a4-93fc-568959642979","Type":"ContainerStarted","Data":"f83d895308ed1bebc6e60e277344011ec8a1011786b4e3de195cef4b895436ca"} Sep 10 13:20:46.557185 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:46.557142 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:20:46.557386 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:46.557345 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vrbmp" podUID="117705e7-b9e7-47ee-a474-d7f74afb23b6" Sep 10 13:20:46.557850 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:46.557792 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:20:46.557850 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:46.557817 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g64cr" Sep 10 13:20:46.558017 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:46.557902 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v2hld" podUID="faa8bfdc-9c79-4c12-8ec9-3cd8639d0793" Sep 10 13:20:46.558017 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:46.557993 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g64cr" podUID="3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597" Sep 10 13:20:46.857829 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:46.857747 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpbx" event={"ID":"a03086c6-fff1-4a62-b2b4-eb4d8266b1e3","Type":"ContainerStarted","Data":"27d632d51aab33c7b4cec9234ecaaa52383f7bff9869fd7cff4b00810e61a6de"} Sep 10 13:20:46.876615 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:46.876563 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2xpbx" podStartSLOduration=1.578434315 podStartE2EDuration="19.876541889s" totalImagesPullingTime="18.298107574s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:20:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:20:46.876043128 +0000 UTC m=+156.770760545" watchObservedRunningTime="2026-09-10 13:20:46.876541889 +0000 UTC m=+156.771259309" Sep 10 13:20:46.876945 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:46.876910 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-4sb6v" podStartSLOduration=7.219830743 podStartE2EDuration="19.876900242s" totalImagesPullingTime="12.657069499s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:20:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:20:45.869979677 +0000 UTC m=+155.764697094" watchObservedRunningTime="2026-09-10 13:20:46.876900242 +0000 UTC m=+156.771617659" Sep 10 13:20:47.809175 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:47.809140 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-2xql8" Sep 10 13:20:47.809798 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:47.809773 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-2xql8" Sep 10 13:20:47.864187 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:47.864154 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" event={"ID":"0db61932-33cb-4fcd-84e6-79282c285bbf","Type":"ContainerStarted","Data":"f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230"} Sep 10 13:20:47.864834 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:47.864314 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-2xql8" Sep 10 13:20:47.864834 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:47.864790 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2xql8" Sep 10 13:20:48.557211 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:48.557177 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:20:48.557211 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:48.557201 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:20:48.557470 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:48.557306 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vrbmp" podUID="117705e7-b9e7-47ee-a474-d7f74afb23b6" Sep 10 13:20:48.557470 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:48.557361 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v2hld" podUID="faa8bfdc-9c79-4c12-8ec9-3cd8639d0793" Sep 10 13:20:48.557470 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:48.557386 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g64cr" Sep 10 13:20:48.557652 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:48.557487 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g64cr" podUID="3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597" Sep 10 13:20:49.873218 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:49.873021 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" event={"ID":"0db61932-33cb-4fcd-84e6-79282c285bbf","Type":"ContainerStarted","Data":"cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3"} Sep 10 13:20:49.873983 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:49.873218 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" podUID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerName="ovn-controller" containerID="cri-o://3fd215ba1d9dd060316b5b3ad36f8a035bc01f4603e770d22b85639c5eb8aed9" gracePeriod=30 Sep 10 13:20:49.873983 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:49.873232 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" podUID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerName="sbdb" containerID="cri-o://f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230" gracePeriod=30 Sep 10 13:20:49.873983 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:49.873247 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" podUID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerName="kube-rbac-proxy-ovn-metrics" containerID="cri-o://3b987da2b6daccdf548c60335fb33085cf2d3d58027b5562894c64b117b0f610" gracePeriod=30 Sep 10 13:20:49.873983 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:49.873355 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:49.873983 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:49.873257 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" podUID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerName="kube-rbac-proxy-node" containerID="cri-o://1d6a5cbf688a0e4d0d9bd3c2538e5ebd4d830b54a409fd038a41a5f0ab92fecf" gracePeriod=30 Sep 10 13:20:49.873983 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:49.873395 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" podUID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerName="nbdb" containerID="cri-o://f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3" gracePeriod=30 Sep 10 13:20:49.873983 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:49.873458 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:49.873983 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:49.873461 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" podUID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerName="northd" containerID="cri-o://ac691669ea7a5766986b91d76178c6feae5635a7049a196d165a08a5a92c1c8a" gracePeriod=30 Sep 10 13:20:49.873983 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:49.873415 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" podUID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerName="ovn-acl-logging" containerID="cri-o://d612f51b0b053f9abd930117864ad6f708f5fad939202d5c0974cf73a1b2b2c9" gracePeriod=30 Sep 10 13:20:49.875810 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:49.875103 2560 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"sb\"\n"] Sep 10 13:20:49.879471 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:49.879326 2560 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"sb\"\n"] Sep 10 13:20:49.880902 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:49.880586 2560 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"sb\"\n"] Sep 10 13:20:49.880902 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:49.880637 2560 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" podUID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerName="sbdb" probeResult="unknown" Sep 10 13:20:49.889626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:49.889249 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" podUID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerName="ovnkube-controller" containerID="cri-o://cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3" gracePeriod=30 Sep 10 13:20:49.914281 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:49.914233 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" podStartSLOduration=6.300885409 podStartE2EDuration="22.914218781s" totalImagesPullingTime="16.613333372s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:20:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:20:49.914011937 +0000 UTC m=+159.808729357" watchObservedRunningTime="2026-09-10 13:20:49.914218781 +0000 UTC m=+159.808936198" Sep 10 13:20:50.119358 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.119332 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t5w8t_0db61932-33cb-4fcd-84e6-79282c285bbf/ovnkube-controller/0.log" Sep 10 13:20:50.120755 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.120734 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t5w8t_0db61932-33cb-4fcd-84e6-79282c285bbf/kube-rbac-proxy-ovn-metrics/0.log" Sep 10 13:20:50.122017 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.121318 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t5w8t_0db61932-33cb-4fcd-84e6-79282c285bbf/kube-rbac-proxy-node/0.log" Sep 10 13:20:50.122017 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.121759 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t5w8t_0db61932-33cb-4fcd-84e6-79282c285bbf/ovn-acl-logging/0.log" Sep 10 13:20:50.122235 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.122222 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t5w8t_0db61932-33cb-4fcd-84e6-79282c285bbf/ovn-controller/0.log" Sep 10 13:20:50.138913 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.138892 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:50.188416 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.188371 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-n86g9"] Sep 10 13:20:50.188554 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.188520 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerName="nbdb" Sep 10 13:20:50.188554 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.188531 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerName="nbdb" Sep 10 13:20:50.188554 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.188539 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerName="ovn-controller" Sep 10 13:20:50.188554 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.188545 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerName="ovn-controller" Sep 10 13:20:50.188554 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.188551 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerName="kube-rbac-proxy-ovn-metrics" Sep 10 13:20:50.188711 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.188558 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerName="kube-rbac-proxy-ovn-metrics" Sep 10 13:20:50.188711 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.188564 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerName="ovnkube-controller" Sep 10 13:20:50.188711 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.188570 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerName="ovnkube-controller" Sep 10 13:20:50.188711 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.188577 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerName="northd" Sep 10 13:20:50.188711 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.188582 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerName="northd" Sep 10 13:20:50.188711 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.188588 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerName="sbdb" Sep 10 13:20:50.188711 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.188593 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerName="sbdb" Sep 10 13:20:50.188711 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.188598 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerName="ovn-acl-logging" Sep 10 13:20:50.188711 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.188603 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerName="ovn-acl-logging" Sep 10 13:20:50.188711 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.188609 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerName="kube-rbac-proxy-node" Sep 10 13:20:50.188711 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.188614 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerName="kube-rbac-proxy-node" Sep 10 13:20:50.188711 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.188638 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerName="ovn-controller" Sep 10 13:20:50.188711 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.188644 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerName="kube-rbac-proxy-ovn-metrics" Sep 10 13:20:50.188711 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.188650 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerName="sbdb" Sep 10 13:20:50.188711 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.188655 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerName="ovnkube-controller" Sep 10 13:20:50.188711 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.188661 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerName="ovn-acl-logging" Sep 10 13:20:50.188711 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.188666 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerName="northd" Sep 10 13:20:50.188711 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.188671 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerName="kube-rbac-proxy-node" Sep 10 13:20:50.188711 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.188676 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerName="nbdb" Sep 10 13:20:50.201029 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.201009 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.216657 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.216630 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lx7pm\" (UniqueName: \"kubernetes.io/projected/0db61932-33cb-4fcd-84e6-79282c285bbf-kube-api-access-lx7pm\") pod \"0db61932-33cb-4fcd-84e6-79282c285bbf\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " Sep 10 13:20:50.216657 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.216657 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"0db61932-33cb-4fcd-84e6-79282c285bbf\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " Sep 10 13:20:50.216859 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.216682 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-slash\") pod \"0db61932-33cb-4fcd-84e6-79282c285bbf\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " Sep 10 13:20:50.216859 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.216698 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-node-log\") pod \"0db61932-33cb-4fcd-84e6-79282c285bbf\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " Sep 10 13:20:50.216859 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.216730 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-node-log" (OuterVolumeSpecName: "node-log") pod "0db61932-33cb-4fcd-84e6-79282c285bbf" (UID: "0db61932-33cb-4fcd-84e6-79282c285bbf"). InnerVolumeSpecName "node-log". PluginName "kubernetes.io/host-path", VolumeGIDValue "" Sep 10 13:20:50.216859 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.216740 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-var-lib-cni-networks-ovn-kubernetes" (OuterVolumeSpecName: "host-var-lib-cni-networks-ovn-kubernetes") pod "0db61932-33cb-4fcd-84e6-79282c285bbf" (UID: "0db61932-33cb-4fcd-84e6-79282c285bbf"). InnerVolumeSpecName "host-var-lib-cni-networks-ovn-kubernetes". PluginName "kubernetes.io/host-path", VolumeGIDValue "" Sep 10 13:20:50.216859 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.216755 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-slash" (OuterVolumeSpecName: "host-slash") pod "0db61932-33cb-4fcd-84e6-79282c285bbf" (UID: "0db61932-33cb-4fcd-84e6-79282c285bbf"). InnerVolumeSpecName "host-slash". PluginName "kubernetes.io/host-path", VolumeGIDValue "" Sep 10 13:20:50.216859 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.216782 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0db61932-33cb-4fcd-84e6-79282c285bbf-ovn-node-metrics-cert\") pod \"0db61932-33cb-4fcd-84e6-79282c285bbf\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " Sep 10 13:20:50.216859 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.216809 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-cni-netd\") pod \"0db61932-33cb-4fcd-84e6-79282c285bbf\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " Sep 10 13:20:50.216859 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.216826 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-run-ovn\") pod \"0db61932-33cb-4fcd-84e6-79282c285bbf\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " Sep 10 13:20:50.216859 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.216846 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-cni-netd" (OuterVolumeSpecName: "host-cni-netd") pod "0db61932-33cb-4fcd-84e6-79282c285bbf" (UID: "0db61932-33cb-4fcd-84e6-79282c285bbf"). InnerVolumeSpecName "host-cni-netd". PluginName "kubernetes.io/host-path", VolumeGIDValue "" Sep 10 13:20:50.216859 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.216851 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-run-netns\") pod \"0db61932-33cb-4fcd-84e6-79282c285bbf\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " Sep 10 13:20:50.217290 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.216873 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-etc-openvswitch\") pod \"0db61932-33cb-4fcd-84e6-79282c285bbf\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " Sep 10 13:20:50.217290 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.216875 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-run-ovn" (OuterVolumeSpecName: "run-ovn") pod "0db61932-33cb-4fcd-84e6-79282c285bbf" (UID: "0db61932-33cb-4fcd-84e6-79282c285bbf"). InnerVolumeSpecName "run-ovn". PluginName "kubernetes.io/host-path", VolumeGIDValue "" Sep 10 13:20:50.217290 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.216892 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-run-netns" (OuterVolumeSpecName: "host-run-netns") pod "0db61932-33cb-4fcd-84e6-79282c285bbf" (UID: "0db61932-33cb-4fcd-84e6-79282c285bbf"). InnerVolumeSpecName "host-run-netns". PluginName "kubernetes.io/host-path", VolumeGIDValue "" Sep 10 13:20:50.217290 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.216917 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-log-socket\") pod \"0db61932-33cb-4fcd-84e6-79282c285bbf\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " Sep 10 13:20:50.217290 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.216917 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-etc-openvswitch" (OuterVolumeSpecName: "etc-openvswitch") pod "0db61932-33cb-4fcd-84e6-79282c285bbf" (UID: "0db61932-33cb-4fcd-84e6-79282c285bbf"). InnerVolumeSpecName "etc-openvswitch". PluginName "kubernetes.io/host-path", VolumeGIDValue "" Sep 10 13:20:50.217290 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.216945 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-cni-bin\") pod \"0db61932-33cb-4fcd-84e6-79282c285bbf\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " Sep 10 13:20:50.217290 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.216945 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-log-socket" (OuterVolumeSpecName: "log-socket") pod "0db61932-33cb-4fcd-84e6-79282c285bbf" (UID: "0db61932-33cb-4fcd-84e6-79282c285bbf"). InnerVolumeSpecName "log-socket". PluginName "kubernetes.io/host-path", VolumeGIDValue "" Sep 10 13:20:50.217290 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.216972 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-run-ovn-kubernetes\") pod \"0db61932-33cb-4fcd-84e6-79282c285bbf\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " Sep 10 13:20:50.217290 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.216993 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-cni-bin" (OuterVolumeSpecName: "host-cni-bin") pod "0db61932-33cb-4fcd-84e6-79282c285bbf" (UID: "0db61932-33cb-4fcd-84e6-79282c285bbf"). InnerVolumeSpecName "host-cni-bin". PluginName "kubernetes.io/host-path", VolumeGIDValue "" Sep 10 13:20:50.217290 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.217005 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-run-openvswitch\") pod \"0db61932-33cb-4fcd-84e6-79282c285bbf\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " Sep 10 13:20:50.217290 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.217028 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-kubelet\") pod \"0db61932-33cb-4fcd-84e6-79282c285bbf\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " Sep 10 13:20:50.217290 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.217049 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-systemd-units\") pod \"0db61932-33cb-4fcd-84e6-79282c285bbf\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " Sep 10 13:20:50.217290 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.217002 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-run-ovn-kubernetes" (OuterVolumeSpecName: "host-run-ovn-kubernetes") pod "0db61932-33cb-4fcd-84e6-79282c285bbf" (UID: "0db61932-33cb-4fcd-84e6-79282c285bbf"). InnerVolumeSpecName "host-run-ovn-kubernetes". PluginName "kubernetes.io/host-path", VolumeGIDValue "" Sep 10 13:20:50.217290 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.217024 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-run-openvswitch" (OuterVolumeSpecName: "run-openvswitch") pod "0db61932-33cb-4fcd-84e6-79282c285bbf" (UID: "0db61932-33cb-4fcd-84e6-79282c285bbf"). InnerVolumeSpecName "run-openvswitch". PluginName "kubernetes.io/host-path", VolumeGIDValue "" Sep 10 13:20:50.217290 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.217048 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-kubelet" (OuterVolumeSpecName: "host-kubelet") pod "0db61932-33cb-4fcd-84e6-79282c285bbf" (UID: "0db61932-33cb-4fcd-84e6-79282c285bbf"). InnerVolumeSpecName "host-kubelet". PluginName "kubernetes.io/host-path", VolumeGIDValue "" Sep 10 13:20:50.217290 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.217076 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0db61932-33cb-4fcd-84e6-79282c285bbf-ovnkube-script-lib\") pod \"0db61932-33cb-4fcd-84e6-79282c285bbf\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " Sep 10 13:20:50.217290 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.217085 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-systemd-units" (OuterVolumeSpecName: "systemd-units") pod "0db61932-33cb-4fcd-84e6-79282c285bbf" (UID: "0db61932-33cb-4fcd-84e6-79282c285bbf"). InnerVolumeSpecName "systemd-units". PluginName "kubernetes.io/host-path", VolumeGIDValue "" Sep 10 13:20:50.218015 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.217106 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0db61932-33cb-4fcd-84e6-79282c285bbf-ovnkube-config\") pod \"0db61932-33cb-4fcd-84e6-79282c285bbf\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " Sep 10 13:20:50.218015 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.217120 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-var-lib-openvswitch\") pod \"0db61932-33cb-4fcd-84e6-79282c285bbf\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " Sep 10 13:20:50.218015 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.217133 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-run-systemd\") pod \"0db61932-33cb-4fcd-84e6-79282c285bbf\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " Sep 10 13:20:50.218015 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.217156 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0db61932-33cb-4fcd-84e6-79282c285bbf-env-overrides\") pod \"0db61932-33cb-4fcd-84e6-79282c285bbf\" (UID: \"0db61932-33cb-4fcd-84e6-79282c285bbf\") " Sep 10 13:20:50.218015 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.217177 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-var-lib-openvswitch" (OuterVolumeSpecName: "var-lib-openvswitch") pod "0db61932-33cb-4fcd-84e6-79282c285bbf" (UID: "0db61932-33cb-4fcd-84e6-79282c285bbf"). InnerVolumeSpecName "var-lib-openvswitch". PluginName "kubernetes.io/host-path", VolumeGIDValue "" Sep 10 13:20:50.218015 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.217305 2560 reconciler_common.go:299] "Volume detached for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-log-socket\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:20:50.218015 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.217324 2560 reconciler_common.go:299] "Volume detached for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-cni-bin\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:20:50.218015 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.217339 2560 reconciler_common.go:299] "Volume detached for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-run-ovn-kubernetes\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:20:50.218015 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.217353 2560 reconciler_common.go:299] "Volume detached for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-run-openvswitch\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:20:50.218015 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.217369 2560 reconciler_common.go:299] "Volume detached for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-kubelet\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:20:50.218015 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.217383 2560 reconciler_common.go:299] "Volume detached for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-systemd-units\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:20:50.218015 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.217396 2560 reconciler_common.go:299] "Volume detached for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-var-lib-openvswitch\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:20:50.218015 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.217432 2560 reconciler_common.go:299] "Volume detached for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-var-lib-cni-networks-ovn-kubernetes\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:20:50.218015 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.217445 2560 reconciler_common.go:299] "Volume detached for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-slash\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:20:50.218015 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.217456 2560 reconciler_common.go:299] "Volume detached for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-node-log\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:20:50.218015 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.217467 2560 reconciler_common.go:299] "Volume detached for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-cni-netd\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:20:50.218015 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.217477 2560 reconciler_common.go:299] "Volume detached for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-run-ovn\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:20:50.218015 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.217488 2560 reconciler_common.go:299] "Volume detached for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-host-run-netns\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:20:50.218015 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.217497 2560 reconciler_common.go:299] "Volume detached for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-etc-openvswitch\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:20:50.218015 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.217558 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0db61932-33cb-4fcd-84e6-79282c285bbf-env-overrides" (OuterVolumeSpecName: "env-overrides") pod "0db61932-33cb-4fcd-84e6-79282c285bbf" (UID: "0db61932-33cb-4fcd-84e6-79282c285bbf"). InnerVolumeSpecName "env-overrides". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:20:50.218015 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.217574 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0db61932-33cb-4fcd-84e6-79282c285bbf-ovnkube-config" (OuterVolumeSpecName: "ovnkube-config") pod "0db61932-33cb-4fcd-84e6-79282c285bbf" (UID: "0db61932-33cb-4fcd-84e6-79282c285bbf"). InnerVolumeSpecName "ovnkube-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:20:50.219026 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.217580 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0db61932-33cb-4fcd-84e6-79282c285bbf-ovnkube-script-lib" (OuterVolumeSpecName: "ovnkube-script-lib") pod "0db61932-33cb-4fcd-84e6-79282c285bbf" (UID: "0db61932-33cb-4fcd-84e6-79282c285bbf"). InnerVolumeSpecName "ovnkube-script-lib". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:20:50.219949 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.219924 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0db61932-33cb-4fcd-84e6-79282c285bbf-ovn-node-metrics-cert" (OuterVolumeSpecName: "ovn-node-metrics-cert") pod "0db61932-33cb-4fcd-84e6-79282c285bbf" (UID: "0db61932-33cb-4fcd-84e6-79282c285bbf"). InnerVolumeSpecName "ovn-node-metrics-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 13:20:50.220158 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.220142 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0db61932-33cb-4fcd-84e6-79282c285bbf-kube-api-access-lx7pm" (OuterVolumeSpecName: "kube-api-access-lx7pm") pod "0db61932-33cb-4fcd-84e6-79282c285bbf" (UID: "0db61932-33cb-4fcd-84e6-79282c285bbf"). InnerVolumeSpecName "kube-api-access-lx7pm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:20:50.220938 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.220915 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-run-systemd" (OuterVolumeSpecName: "run-systemd") pod "0db61932-33cb-4fcd-84e6-79282c285bbf" (UID: "0db61932-33cb-4fcd-84e6-79282c285bbf"). InnerVolumeSpecName "run-systemd". PluginName "kubernetes.io/host-path", VolumeGIDValue "" Sep 10 13:20:50.318368 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.318273 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/edcc06ac-ef52-48ca-b6db-7c889342d723-ovnkube-script-lib\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.318368 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.318310 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-host-kubelet\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.318368 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.318341 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-host-slash\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.318368 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.318365 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/edcc06ac-ef52-48ca-b6db-7c889342d723-env-overrides\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.318617 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.318390 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-etc-openvswitch\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.318617 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.318424 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-host-cni-netd\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.318617 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.318506 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-run-openvswitch\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.318617 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.318529 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-host-run-ovn-kubernetes\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.318617 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.318555 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-systemd-units\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.318617 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.318575 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-node-log\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.318617 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.318591 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-host-cni-bin\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.318617 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.318612 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-run-ovn\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.318881 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.318638 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/edcc06ac-ef52-48ca-b6db-7c889342d723-ovnkube-config\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.318881 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.318684 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-host-run-netns\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.318881 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.318699 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-run-systemd\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.318881 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.318717 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-var-lib-openvswitch\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.318881 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.318732 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.318881 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.318747 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/edcc06ac-ef52-48ca-b6db-7c889342d723-ovn-node-metrics-cert\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.318881 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.318761 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjwfc\" (UniqueName: \"kubernetes.io/projected/edcc06ac-ef52-48ca-b6db-7c889342d723-kube-api-access-hjwfc\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.318881 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.318786 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-log-socket\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.318881 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.318848 2560 reconciler_common.go:299] "Volume detached for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0db61932-33cb-4fcd-84e6-79282c285bbf-ovnkube-config\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:20:50.319132 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.318889 2560 reconciler_common.go:299] "Volume detached for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0db61932-33cb-4fcd-84e6-79282c285bbf-run-systemd\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:20:50.319132 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.318906 2560 reconciler_common.go:299] "Volume detached for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0db61932-33cb-4fcd-84e6-79282c285bbf-env-overrides\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:20:50.319132 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.318916 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lx7pm\" (UniqueName: \"kubernetes.io/projected/0db61932-33cb-4fcd-84e6-79282c285bbf-kube-api-access-lx7pm\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:20:50.319132 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.318926 2560 reconciler_common.go:299] "Volume detached for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0db61932-33cb-4fcd-84e6-79282c285bbf-ovn-node-metrics-cert\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:20:50.319132 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.318937 2560 reconciler_common.go:299] "Volume detached for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0db61932-33cb-4fcd-84e6-79282c285bbf-ovnkube-script-lib\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:20:50.419235 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.419197 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-run-openvswitch\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.419235 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.419236 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-host-run-ovn-kubernetes\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.419422 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.419254 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-systemd-units\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.419422 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.419276 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-node-log\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.419422 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.419300 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-host-cni-bin\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.419422 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.419317 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-run-openvswitch\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.419422 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.419321 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-host-run-ovn-kubernetes\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.419422 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.419339 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-node-log\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.419422 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.419328 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-run-ovn\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.419422 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.419329 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-systemd-units\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.419422 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.419364 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-run-ovn\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.419422 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.419362 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-host-cni-bin\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.419422 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.419391 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/edcc06ac-ef52-48ca-b6db-7c889342d723-ovnkube-config\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.419815 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.419438 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-host-run-netns\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.419815 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.419460 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-run-systemd\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.419815 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.419477 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-var-lib-openvswitch\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.419815 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.419492 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.419815 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.419509 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/edcc06ac-ef52-48ca-b6db-7c889342d723-ovn-node-metrics-cert\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.419815 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.419534 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjwfc\" (UniqueName: \"kubernetes.io/projected/edcc06ac-ef52-48ca-b6db-7c889342d723-kube-api-access-hjwfc\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.419815 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.419543 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-run-systemd\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.419815 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.419560 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-log-socket\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.419815 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.419568 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.419815 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.419579 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-host-run-netns\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.419815 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.419585 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/edcc06ac-ef52-48ca-b6db-7c889342d723-ovnkube-script-lib\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.419815 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.419609 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-host-kubelet\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.419815 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.419616 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-log-socket\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.419815 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.419544 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-var-lib-openvswitch\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.419815 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.419656 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-host-slash\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.419815 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.419660 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-host-kubelet\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.419815 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.419688 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/edcc06ac-ef52-48ca-b6db-7c889342d723-env-overrides\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.420431 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.419691 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-host-slash\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.420431 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.419724 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-etc-openvswitch\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.420431 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.419747 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-host-cni-netd\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.420431 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.419810 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-host-cni-netd\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.420431 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.419803 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/edcc06ac-ef52-48ca-b6db-7c889342d723-etc-openvswitch\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.420431 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.420039 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/edcc06ac-ef52-48ca-b6db-7c889342d723-ovnkube-config\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.420431 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.420112 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/edcc06ac-ef52-48ca-b6db-7c889342d723-env-overrides\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.420431 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.420121 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/edcc06ac-ef52-48ca-b6db-7c889342d723-ovnkube-script-lib\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.421899 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.421880 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/edcc06ac-ef52-48ca-b6db-7c889342d723-ovn-node-metrics-cert\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.429120 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.429100 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjwfc\" (UniqueName: \"kubernetes.io/projected/edcc06ac-ef52-48ca-b6db-7c889342d723-kube-api-access-hjwfc\") pod \"ovnkube-node-n86g9\" (UID: \"edcc06ac-ef52-48ca-b6db-7c889342d723\") " pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.501479 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:50.501445 2560 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 10 13:20:50.527279 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.527254 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:50.557317 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.557293 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g64cr" Sep 10 13:20:50.557472 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.557357 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:20:50.557472 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:50.557368 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g64cr" podUID="3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597" Sep 10 13:20:50.557567 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:50.557473 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vrbmp" podUID="117705e7-b9e7-47ee-a474-d7f74afb23b6" Sep 10 13:20:50.557567 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.557514 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:20:50.557567 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:50.557563 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v2hld" podUID="faa8bfdc-9c79-4c12-8ec9-3cd8639d0793" Sep 10 13:20:50.564866 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:20:50.564846 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedcc06ac_ef52_48ca_b6db_7c889342d723.slice/crio-2cfbfa1c2ed228bc46a265eaf6dab40561800509219c47c1ac3150410c8c740d WatchSource:0}: Error finding container 2cfbfa1c2ed228bc46a265eaf6dab40561800509219c47c1ac3150410c8c740d: Status 404 returned error can't find the container with id 2cfbfa1c2ed228bc46a265eaf6dab40561800509219c47c1ac3150410c8c740d Sep 10 13:20:50.876151 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.875990 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t5w8t_0db61932-33cb-4fcd-84e6-79282c285bbf/ovnkube-controller/0.log" Sep 10 13:20:50.877379 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.877362 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t5w8t_0db61932-33cb-4fcd-84e6-79282c285bbf/kube-rbac-proxy-ovn-metrics/0.log" Sep 10 13:20:50.877799 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.877783 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t5w8t_0db61932-33cb-4fcd-84e6-79282c285bbf/kube-rbac-proxy-node/0.log" Sep 10 13:20:50.878165 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.878153 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t5w8t_0db61932-33cb-4fcd-84e6-79282c285bbf/ovn-acl-logging/0.log" Sep 10 13:20:50.878546 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.878532 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t5w8t_0db61932-33cb-4fcd-84e6-79282c285bbf/ovn-controller/0.log" Sep 10 13:20:50.878633 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.878570 2560 generic.go:356] "Generic (PLEG): container finished" podID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerID="cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3" exitCode=143 Sep 10 13:20:50.878633 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.878587 2560 generic.go:356] "Generic (PLEG): container finished" podID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerID="f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230" exitCode=0 Sep 10 13:20:50.878633 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.878596 2560 generic.go:356] "Generic (PLEG): container finished" podID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerID="f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3" exitCode=0 Sep 10 13:20:50.878633 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.878604 2560 generic.go:356] "Generic (PLEG): container finished" podID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerID="ac691669ea7a5766986b91d76178c6feae5635a7049a196d165a08a5a92c1c8a" exitCode=0 Sep 10 13:20:50.878633 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.878612 2560 generic.go:356] "Generic (PLEG): container finished" podID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerID="3b987da2b6daccdf548c60335fb33085cf2d3d58027b5562894c64b117b0f610" exitCode=143 Sep 10 13:20:50.878633 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.878620 2560 generic.go:356] "Generic (PLEG): container finished" podID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerID="1d6a5cbf688a0e4d0d9bd3c2538e5ebd4d830b54a409fd038a41a5f0ab92fecf" exitCode=143 Sep 10 13:20:50.878633 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.878627 2560 generic.go:356] "Generic (PLEG): container finished" podID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerID="d612f51b0b053f9abd930117864ad6f708f5fad939202d5c0974cf73a1b2b2c9" exitCode=143 Sep 10 13:20:50.878633 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.878635 2560 generic.go:356] "Generic (PLEG): container finished" podID="0db61932-33cb-4fcd-84e6-79282c285bbf" containerID="3fd215ba1d9dd060316b5b3ad36f8a035bc01f4603e770d22b85639c5eb8aed9" exitCode=143 Sep 10 13:20:50.878931 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.878655 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" Sep 10 13:20:50.878931 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.878648 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" event={"ID":"0db61932-33cb-4fcd-84e6-79282c285bbf","Type":"ContainerDied","Data":"cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3"} Sep 10 13:20:50.878931 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.878809 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" event={"ID":"0db61932-33cb-4fcd-84e6-79282c285bbf","Type":"ContainerDied","Data":"f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230"} Sep 10 13:20:50.878931 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.878853 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" event={"ID":"0db61932-33cb-4fcd-84e6-79282c285bbf","Type":"ContainerDied","Data":"f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3"} Sep 10 13:20:50.878931 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.878887 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" event={"ID":"0db61932-33cb-4fcd-84e6-79282c285bbf","Type":"ContainerDied","Data":"ac691669ea7a5766986b91d76178c6feae5635a7049a196d165a08a5a92c1c8a"} Sep 10 13:20:50.878931 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.878902 2560 scope.go:117] "RemoveContainer" containerID="cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3" Sep 10 13:20:50.878931 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.878926 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" event={"ID":"0db61932-33cb-4fcd-84e6-79282c285bbf","Type":"ContainerDied","Data":"3b987da2b6daccdf548c60335fb33085cf2d3d58027b5562894c64b117b0f610"} Sep 10 13:20:50.879226 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.878962 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" event={"ID":"0db61932-33cb-4fcd-84e6-79282c285bbf","Type":"ContainerDied","Data":"1d6a5cbf688a0e4d0d9bd3c2538e5ebd4d830b54a409fd038a41a5f0ab92fecf"} Sep 10 13:20:50.879226 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.879012 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" event={"ID":"0db61932-33cb-4fcd-84e6-79282c285bbf","Type":"ContainerDied","Data":"d612f51b0b053f9abd930117864ad6f708f5fad939202d5c0974cf73a1b2b2c9"} Sep 10 13:20:50.879226 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.879068 2560 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"ac691669ea7a5766986b91d76178c6feae5635a7049a196d165a08a5a92c1c8a"} Sep 10 13:20:50.879226 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.879097 2560 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"3b987da2b6daccdf548c60335fb33085cf2d3d58027b5562894c64b117b0f610"} Sep 10 13:20:50.879226 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.879104 2560 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"1d6a5cbf688a0e4d0d9bd3c2538e5ebd4d830b54a409fd038a41a5f0ab92fecf"} Sep 10 13:20:50.879226 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.879111 2560 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"d612f51b0b053f9abd930117864ad6f708f5fad939202d5c0974cf73a1b2b2c9"} Sep 10 13:20:50.879226 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.879117 2560 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"3fd215ba1d9dd060316b5b3ad36f8a035bc01f4603e770d22b85639c5eb8aed9"} Sep 10 13:20:50.879226 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.879128 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" event={"ID":"0db61932-33cb-4fcd-84e6-79282c285bbf","Type":"ContainerDied","Data":"3fd215ba1d9dd060316b5b3ad36f8a035bc01f4603e770d22b85639c5eb8aed9"} Sep 10 13:20:50.879226 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.879164 2560 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3"} Sep 10 13:20:50.879226 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.879171 2560 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230"} Sep 10 13:20:50.879226 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.879175 2560 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3"} Sep 10 13:20:50.879226 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.879179 2560 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"ac691669ea7a5766986b91d76178c6feae5635a7049a196d165a08a5a92c1c8a"} Sep 10 13:20:50.879226 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.879183 2560 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"3b987da2b6daccdf548c60335fb33085cf2d3d58027b5562894c64b117b0f610"} Sep 10 13:20:50.879226 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.879187 2560 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"1d6a5cbf688a0e4d0d9bd3c2538e5ebd4d830b54a409fd038a41a5f0ab92fecf"} Sep 10 13:20:50.879226 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.879192 2560 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"d612f51b0b053f9abd930117864ad6f708f5fad939202d5c0974cf73a1b2b2c9"} Sep 10 13:20:50.879226 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.879196 2560 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"3fd215ba1d9dd060316b5b3ad36f8a035bc01f4603e770d22b85639c5eb8aed9"} Sep 10 13:20:50.879226 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.879203 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t5w8t" event={"ID":"0db61932-33cb-4fcd-84e6-79282c285bbf","Type":"ContainerDied","Data":"14fd596d470db1c9ae38b85eb33b88527b6572430bbab1903f02a876334e614e"} Sep 10 13:20:50.879226 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.879238 2560 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3"} Sep 10 13:20:50.879974 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.879244 2560 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230"} Sep 10 13:20:50.879974 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.879248 2560 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3"} Sep 10 13:20:50.879974 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.879254 2560 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"ac691669ea7a5766986b91d76178c6feae5635a7049a196d165a08a5a92c1c8a"} Sep 10 13:20:50.879974 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.879260 2560 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"3b987da2b6daccdf548c60335fb33085cf2d3d58027b5562894c64b117b0f610"} Sep 10 13:20:50.879974 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.879265 2560 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"1d6a5cbf688a0e4d0d9bd3c2538e5ebd4d830b54a409fd038a41a5f0ab92fecf"} Sep 10 13:20:50.879974 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.879271 2560 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"d612f51b0b053f9abd930117864ad6f708f5fad939202d5c0974cf73a1b2b2c9"} Sep 10 13:20:50.879974 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.879277 2560 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"3fd215ba1d9dd060316b5b3ad36f8a035bc01f4603e770d22b85639c5eb8aed9"} Sep 10 13:20:50.880903 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.880887 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" event={"ID":"edcc06ac-ef52-48ca-b6db-7c889342d723","Type":"ContainerStarted","Data":"81434f712c79e8e0bd0b228386a456e9c5d1b7b962e3913396886fe7999bad5e"} Sep 10 13:20:50.880977 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.880960 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" event={"ID":"edcc06ac-ef52-48ca-b6db-7c889342d723","Type":"ContainerStarted","Data":"d7e5ad71323e12adf3c97a7911cd2404a277af4c254bbfe82731006c4ed3e688"} Sep 10 13:20:50.881029 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.881013 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" event={"ID":"edcc06ac-ef52-48ca-b6db-7c889342d723","Type":"ContainerStarted","Data":"2cfbfa1c2ed228bc46a265eaf6dab40561800509219c47c1ac3150410c8c740d"} Sep 10 13:20:50.882677 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.882657 2560 generic.go:356] "Generic (PLEG): container finished" podID="cc3dddf2-363d-4c1d-824f-aaca292a88f0" containerID="b9bc6886e77227cbca24c365f7edd0200912f8bd37b5350385931423a326e386" exitCode=0 Sep 10 13:20:50.882756 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.882690 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-28xnt" event={"ID":"cc3dddf2-363d-4c1d-824f-aaca292a88f0","Type":"ContainerDied","Data":"b9bc6886e77227cbca24c365f7edd0200912f8bd37b5350385931423a326e386"} Sep 10 13:20:50.888174 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.888155 2560 scope.go:117] "RemoveContainer" containerID="f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230" Sep 10 13:20:50.895374 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.895361 2560 scope.go:117] "RemoveContainer" containerID="f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3" Sep 10 13:20:50.911080 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.911055 2560 scope.go:117] "RemoveContainer" containerID="ac691669ea7a5766986b91d76178c6feae5635a7049a196d165a08a5a92c1c8a" Sep 10 13:20:50.916777 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.916758 2560 scope.go:117] "RemoveContainer" containerID="3b987da2b6daccdf548c60335fb33085cf2d3d58027b5562894c64b117b0f610" Sep 10 13:20:50.922163 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.922149 2560 scope.go:117] "RemoveContainer" containerID="1d6a5cbf688a0e4d0d9bd3c2538e5ebd4d830b54a409fd038a41a5f0ab92fecf" Sep 10 13:20:50.927569 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.927554 2560 scope.go:117] "RemoveContainer" containerID="d612f51b0b053f9abd930117864ad6f708f5fad939202d5c0974cf73a1b2b2c9" Sep 10 13:20:50.929458 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.929438 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-t5w8t"] Sep 10 13:20:50.933119 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.933096 2560 scope.go:117] "RemoveContainer" containerID="3fd215ba1d9dd060316b5b3ad36f8a035bc01f4603e770d22b85639c5eb8aed9" Sep 10 13:20:50.939187 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.939171 2560 scope.go:117] "RemoveContainer" containerID="cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3" Sep 10 13:20:50.939502 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:50.939480 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3\": container with ID starting with cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3 not found: ID does not exist" containerID="cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3" Sep 10 13:20:50.939553 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.939515 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3"} err="failed to get container status \"cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3\": rpc error: code = NotFound desc = could not find container \"cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3\": container with ID starting with cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3 not found: ID does not exist" Sep 10 13:20:50.939553 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.939538 2560 scope.go:117] "RemoveContainer" containerID="f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230" Sep 10 13:20:50.939810 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:50.939795 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230\": container with ID starting with f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230 not found: ID does not exist" containerID="f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230" Sep 10 13:20:50.939852 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.939816 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230"} err="failed to get container status \"f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230\": rpc error: code = NotFound desc = could not find container \"f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230\": container with ID starting with f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230 not found: ID does not exist" Sep 10 13:20:50.939852 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.939829 2560 scope.go:117] "RemoveContainer" containerID="f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3" Sep 10 13:20:50.940073 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:50.940055 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3\": container with ID starting with f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3 not found: ID does not exist" containerID="f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3" Sep 10 13:20:50.940111 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.940079 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3"} err="failed to get container status \"f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3\": rpc error: code = NotFound desc = could not find container \"f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3\": container with ID starting with f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3 not found: ID does not exist" Sep 10 13:20:50.940111 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.940094 2560 scope.go:117] "RemoveContainer" containerID="ac691669ea7a5766986b91d76178c6feae5635a7049a196d165a08a5a92c1c8a" Sep 10 13:20:50.940305 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:50.940292 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac691669ea7a5766986b91d76178c6feae5635a7049a196d165a08a5a92c1c8a\": container with ID starting with ac691669ea7a5766986b91d76178c6feae5635a7049a196d165a08a5a92c1c8a not found: ID does not exist" containerID="ac691669ea7a5766986b91d76178c6feae5635a7049a196d165a08a5a92c1c8a" Sep 10 13:20:50.940349 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.940308 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac691669ea7a5766986b91d76178c6feae5635a7049a196d165a08a5a92c1c8a"} err="failed to get container status \"ac691669ea7a5766986b91d76178c6feae5635a7049a196d165a08a5a92c1c8a\": rpc error: code = NotFound desc = could not find container \"ac691669ea7a5766986b91d76178c6feae5635a7049a196d165a08a5a92c1c8a\": container with ID starting with ac691669ea7a5766986b91d76178c6feae5635a7049a196d165a08a5a92c1c8a not found: ID does not exist" Sep 10 13:20:50.940349 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.940320 2560 scope.go:117] "RemoveContainer" containerID="3b987da2b6daccdf548c60335fb33085cf2d3d58027b5562894c64b117b0f610" Sep 10 13:20:50.940566 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:50.940550 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b987da2b6daccdf548c60335fb33085cf2d3d58027b5562894c64b117b0f610\": container with ID starting with 3b987da2b6daccdf548c60335fb33085cf2d3d58027b5562894c64b117b0f610 not found: ID does not exist" containerID="3b987da2b6daccdf548c60335fb33085cf2d3d58027b5562894c64b117b0f610" Sep 10 13:20:50.940602 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.940572 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b987da2b6daccdf548c60335fb33085cf2d3d58027b5562894c64b117b0f610"} err="failed to get container status \"3b987da2b6daccdf548c60335fb33085cf2d3d58027b5562894c64b117b0f610\": rpc error: code = NotFound desc = could not find container \"3b987da2b6daccdf548c60335fb33085cf2d3d58027b5562894c64b117b0f610\": container with ID starting with 3b987da2b6daccdf548c60335fb33085cf2d3d58027b5562894c64b117b0f610 not found: ID does not exist" Sep 10 13:20:50.940602 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.940585 2560 scope.go:117] "RemoveContainer" containerID="1d6a5cbf688a0e4d0d9bd3c2538e5ebd4d830b54a409fd038a41a5f0ab92fecf" Sep 10 13:20:50.940806 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:50.940791 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d6a5cbf688a0e4d0d9bd3c2538e5ebd4d830b54a409fd038a41a5f0ab92fecf\": container with ID starting with 1d6a5cbf688a0e4d0d9bd3c2538e5ebd4d830b54a409fd038a41a5f0ab92fecf not found: ID does not exist" containerID="1d6a5cbf688a0e4d0d9bd3c2538e5ebd4d830b54a409fd038a41a5f0ab92fecf" Sep 10 13:20:50.940850 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.940809 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d6a5cbf688a0e4d0d9bd3c2538e5ebd4d830b54a409fd038a41a5f0ab92fecf"} err="failed to get container status \"1d6a5cbf688a0e4d0d9bd3c2538e5ebd4d830b54a409fd038a41a5f0ab92fecf\": rpc error: code = NotFound desc = could not find container \"1d6a5cbf688a0e4d0d9bd3c2538e5ebd4d830b54a409fd038a41a5f0ab92fecf\": container with ID starting with 1d6a5cbf688a0e4d0d9bd3c2538e5ebd4d830b54a409fd038a41a5f0ab92fecf not found: ID does not exist" Sep 10 13:20:50.940850 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.940821 2560 scope.go:117] "RemoveContainer" containerID="d612f51b0b053f9abd930117864ad6f708f5fad939202d5c0974cf73a1b2b2c9" Sep 10 13:20:50.941011 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:50.940996 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d612f51b0b053f9abd930117864ad6f708f5fad939202d5c0974cf73a1b2b2c9\": container with ID starting with d612f51b0b053f9abd930117864ad6f708f5fad939202d5c0974cf73a1b2b2c9 not found: ID does not exist" containerID="d612f51b0b053f9abd930117864ad6f708f5fad939202d5c0974cf73a1b2b2c9" Sep 10 13:20:50.941044 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.941014 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d612f51b0b053f9abd930117864ad6f708f5fad939202d5c0974cf73a1b2b2c9"} err="failed to get container status \"d612f51b0b053f9abd930117864ad6f708f5fad939202d5c0974cf73a1b2b2c9\": rpc error: code = NotFound desc = could not find container \"d612f51b0b053f9abd930117864ad6f708f5fad939202d5c0974cf73a1b2b2c9\": container with ID starting with d612f51b0b053f9abd930117864ad6f708f5fad939202d5c0974cf73a1b2b2c9 not found: ID does not exist" Sep 10 13:20:50.941044 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.941026 2560 scope.go:117] "RemoveContainer" containerID="3fd215ba1d9dd060316b5b3ad36f8a035bc01f4603e770d22b85639c5eb8aed9" Sep 10 13:20:50.941224 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:50.941206 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fd215ba1d9dd060316b5b3ad36f8a035bc01f4603e770d22b85639c5eb8aed9\": container with ID starting with 3fd215ba1d9dd060316b5b3ad36f8a035bc01f4603e770d22b85639c5eb8aed9 not found: ID does not exist" containerID="3fd215ba1d9dd060316b5b3ad36f8a035bc01f4603e770d22b85639c5eb8aed9" Sep 10 13:20:50.941265 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.941229 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fd215ba1d9dd060316b5b3ad36f8a035bc01f4603e770d22b85639c5eb8aed9"} err="failed to get container status \"3fd215ba1d9dd060316b5b3ad36f8a035bc01f4603e770d22b85639c5eb8aed9\": rpc error: code = NotFound desc = could not find container \"3fd215ba1d9dd060316b5b3ad36f8a035bc01f4603e770d22b85639c5eb8aed9\": container with ID starting with 3fd215ba1d9dd060316b5b3ad36f8a035bc01f4603e770d22b85639c5eb8aed9 not found: ID does not exist" Sep 10 13:20:50.941265 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.941241 2560 scope.go:117] "RemoveContainer" containerID="cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3" Sep 10 13:20:50.941487 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.941469 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3"} err="failed to get container status \"cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3\": rpc error: code = NotFound desc = could not find container \"cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3\": container with ID starting with cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3 not found: ID does not exist" Sep 10 13:20:50.941525 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.941488 2560 scope.go:117] "RemoveContainer" containerID="f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230" Sep 10 13:20:50.941686 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.941671 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230"} err="failed to get container status \"f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230\": rpc error: code = NotFound desc = could not find container \"f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230\": container with ID starting with f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230 not found: ID does not exist" Sep 10 13:20:50.941728 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.941685 2560 scope.go:117] "RemoveContainer" containerID="f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3" Sep 10 13:20:50.941892 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.941877 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3"} err="failed to get container status \"f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3\": rpc error: code = NotFound desc = could not find container \"f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3\": container with ID starting with f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3 not found: ID does not exist" Sep 10 13:20:50.941927 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.941892 2560 scope.go:117] "RemoveContainer" containerID="ac691669ea7a5766986b91d76178c6feae5635a7049a196d165a08a5a92c1c8a" Sep 10 13:20:50.942098 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.942082 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac691669ea7a5766986b91d76178c6feae5635a7049a196d165a08a5a92c1c8a"} err="failed to get container status \"ac691669ea7a5766986b91d76178c6feae5635a7049a196d165a08a5a92c1c8a\": rpc error: code = NotFound desc = could not find container \"ac691669ea7a5766986b91d76178c6feae5635a7049a196d165a08a5a92c1c8a\": container with ID starting with ac691669ea7a5766986b91d76178c6feae5635a7049a196d165a08a5a92c1c8a not found: ID does not exist" Sep 10 13:20:50.942142 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.942099 2560 scope.go:117] "RemoveContainer" containerID="3b987da2b6daccdf548c60335fb33085cf2d3d58027b5562894c64b117b0f610" Sep 10 13:20:50.942332 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.942315 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b987da2b6daccdf548c60335fb33085cf2d3d58027b5562894c64b117b0f610"} err="failed to get container status \"3b987da2b6daccdf548c60335fb33085cf2d3d58027b5562894c64b117b0f610\": rpc error: code = NotFound desc = could not find container \"3b987da2b6daccdf548c60335fb33085cf2d3d58027b5562894c64b117b0f610\": container with ID starting with 3b987da2b6daccdf548c60335fb33085cf2d3d58027b5562894c64b117b0f610 not found: ID does not exist" Sep 10 13:20:50.942370 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.942333 2560 scope.go:117] "RemoveContainer" containerID="1d6a5cbf688a0e4d0d9bd3c2538e5ebd4d830b54a409fd038a41a5f0ab92fecf" Sep 10 13:20:50.942580 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.942564 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d6a5cbf688a0e4d0d9bd3c2538e5ebd4d830b54a409fd038a41a5f0ab92fecf"} err="failed to get container status \"1d6a5cbf688a0e4d0d9bd3c2538e5ebd4d830b54a409fd038a41a5f0ab92fecf\": rpc error: code = NotFound desc = could not find container \"1d6a5cbf688a0e4d0d9bd3c2538e5ebd4d830b54a409fd038a41a5f0ab92fecf\": container with ID starting with 1d6a5cbf688a0e4d0d9bd3c2538e5ebd4d830b54a409fd038a41a5f0ab92fecf not found: ID does not exist" Sep 10 13:20:50.942616 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.942580 2560 scope.go:117] "RemoveContainer" containerID="d612f51b0b053f9abd930117864ad6f708f5fad939202d5c0974cf73a1b2b2c9" Sep 10 13:20:50.942798 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.942779 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d612f51b0b053f9abd930117864ad6f708f5fad939202d5c0974cf73a1b2b2c9"} err="failed to get container status \"d612f51b0b053f9abd930117864ad6f708f5fad939202d5c0974cf73a1b2b2c9\": rpc error: code = NotFound desc = could not find container \"d612f51b0b053f9abd930117864ad6f708f5fad939202d5c0974cf73a1b2b2c9\": container with ID starting with d612f51b0b053f9abd930117864ad6f708f5fad939202d5c0974cf73a1b2b2c9 not found: ID does not exist" Sep 10 13:20:50.942844 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.942799 2560 scope.go:117] "RemoveContainer" containerID="3fd215ba1d9dd060316b5b3ad36f8a035bc01f4603e770d22b85639c5eb8aed9" Sep 10 13:20:50.943019 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.943002 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fd215ba1d9dd060316b5b3ad36f8a035bc01f4603e770d22b85639c5eb8aed9"} err="failed to get container status \"3fd215ba1d9dd060316b5b3ad36f8a035bc01f4603e770d22b85639c5eb8aed9\": rpc error: code = NotFound desc = could not find container \"3fd215ba1d9dd060316b5b3ad36f8a035bc01f4603e770d22b85639c5eb8aed9\": container with ID starting with 3fd215ba1d9dd060316b5b3ad36f8a035bc01f4603e770d22b85639c5eb8aed9 not found: ID does not exist" Sep 10 13:20:50.943077 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.943018 2560 scope.go:117] "RemoveContainer" containerID="cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3" Sep 10 13:20:50.943234 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.943215 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3"} err="failed to get container status \"cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3\": rpc error: code = NotFound desc = could not find container \"cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3\": container with ID starting with cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3 not found: ID does not exist" Sep 10 13:20:50.943275 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.943235 2560 scope.go:117] "RemoveContainer" containerID="f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230" Sep 10 13:20:50.943531 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.943508 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230"} err="failed to get container status \"f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230\": rpc error: code = NotFound desc = could not find container \"f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230\": container with ID starting with f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230 not found: ID does not exist" Sep 10 13:20:50.943577 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.943534 2560 scope.go:117] "RemoveContainer" containerID="f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3" Sep 10 13:20:50.943795 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.943778 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3"} err="failed to get container status \"f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3\": rpc error: code = NotFound desc = could not find container \"f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3\": container with ID starting with f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3 not found: ID does not exist" Sep 10 13:20:50.943840 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.943797 2560 scope.go:117] "RemoveContainer" containerID="ac691669ea7a5766986b91d76178c6feae5635a7049a196d165a08a5a92c1c8a" Sep 10 13:20:50.944077 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.944049 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac691669ea7a5766986b91d76178c6feae5635a7049a196d165a08a5a92c1c8a"} err="failed to get container status \"ac691669ea7a5766986b91d76178c6feae5635a7049a196d165a08a5a92c1c8a\": rpc error: code = NotFound desc = could not find container \"ac691669ea7a5766986b91d76178c6feae5635a7049a196d165a08a5a92c1c8a\": container with ID starting with ac691669ea7a5766986b91d76178c6feae5635a7049a196d165a08a5a92c1c8a not found: ID does not exist" Sep 10 13:20:50.944145 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.944078 2560 scope.go:117] "RemoveContainer" containerID="3b987da2b6daccdf548c60335fb33085cf2d3d58027b5562894c64b117b0f610" Sep 10 13:20:50.944361 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.944345 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-t5w8t"] Sep 10 13:20:50.944431 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.944349 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b987da2b6daccdf548c60335fb33085cf2d3d58027b5562894c64b117b0f610"} err="failed to get container status \"3b987da2b6daccdf548c60335fb33085cf2d3d58027b5562894c64b117b0f610\": rpc error: code = NotFound desc = could not find container \"3b987da2b6daccdf548c60335fb33085cf2d3d58027b5562894c64b117b0f610\": container with ID starting with 3b987da2b6daccdf548c60335fb33085cf2d3d58027b5562894c64b117b0f610 not found: ID does not exist" Sep 10 13:20:50.944431 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.944387 2560 scope.go:117] "RemoveContainer" containerID="1d6a5cbf688a0e4d0d9bd3c2538e5ebd4d830b54a409fd038a41a5f0ab92fecf" Sep 10 13:20:50.944663 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.944647 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d6a5cbf688a0e4d0d9bd3c2538e5ebd4d830b54a409fd038a41a5f0ab92fecf"} err="failed to get container status \"1d6a5cbf688a0e4d0d9bd3c2538e5ebd4d830b54a409fd038a41a5f0ab92fecf\": rpc error: code = NotFound desc = could not find container \"1d6a5cbf688a0e4d0d9bd3c2538e5ebd4d830b54a409fd038a41a5f0ab92fecf\": container with ID starting with 1d6a5cbf688a0e4d0d9bd3c2538e5ebd4d830b54a409fd038a41a5f0ab92fecf not found: ID does not exist" Sep 10 13:20:50.944716 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.944663 2560 scope.go:117] "RemoveContainer" containerID="d612f51b0b053f9abd930117864ad6f708f5fad939202d5c0974cf73a1b2b2c9" Sep 10 13:20:50.945147 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.945054 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d612f51b0b053f9abd930117864ad6f708f5fad939202d5c0974cf73a1b2b2c9"} err="failed to get container status \"d612f51b0b053f9abd930117864ad6f708f5fad939202d5c0974cf73a1b2b2c9\": rpc error: code = NotFound desc = could not find container \"d612f51b0b053f9abd930117864ad6f708f5fad939202d5c0974cf73a1b2b2c9\": container with ID starting with d612f51b0b053f9abd930117864ad6f708f5fad939202d5c0974cf73a1b2b2c9 not found: ID does not exist" Sep 10 13:20:50.945147 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.945078 2560 scope.go:117] "RemoveContainer" containerID="3fd215ba1d9dd060316b5b3ad36f8a035bc01f4603e770d22b85639c5eb8aed9" Sep 10 13:20:50.947235 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.946218 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fd215ba1d9dd060316b5b3ad36f8a035bc01f4603e770d22b85639c5eb8aed9"} err="failed to get container status \"3fd215ba1d9dd060316b5b3ad36f8a035bc01f4603e770d22b85639c5eb8aed9\": rpc error: code = NotFound desc = could not find container \"3fd215ba1d9dd060316b5b3ad36f8a035bc01f4603e770d22b85639c5eb8aed9\": container with ID starting with 3fd215ba1d9dd060316b5b3ad36f8a035bc01f4603e770d22b85639c5eb8aed9 not found: ID does not exist" Sep 10 13:20:50.947235 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.946242 2560 scope.go:117] "RemoveContainer" containerID="cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3" Sep 10 13:20:50.947235 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.947103 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3"} err="failed to get container status \"cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3\": rpc error: code = NotFound desc = could not find container \"cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3\": container with ID starting with cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3 not found: ID does not exist" Sep 10 13:20:50.947235 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.947141 2560 scope.go:117] "RemoveContainer" containerID="f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230" Sep 10 13:20:50.950524 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.949719 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230"} err="failed to get container status \"f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230\": rpc error: code = NotFound desc = could not find container \"f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230\": container with ID starting with f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230 not found: ID does not exist" Sep 10 13:20:50.950524 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.949748 2560 scope.go:117] "RemoveContainer" containerID="f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3" Sep 10 13:20:50.951100 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.951072 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3"} err="failed to get container status \"f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3\": rpc error: code = NotFound desc = could not find container \"f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3\": container with ID starting with f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3 not found: ID does not exist" Sep 10 13:20:50.951100 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.951098 2560 scope.go:117] "RemoveContainer" containerID="ac691669ea7a5766986b91d76178c6feae5635a7049a196d165a08a5a92c1c8a" Sep 10 13:20:50.951429 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.951389 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac691669ea7a5766986b91d76178c6feae5635a7049a196d165a08a5a92c1c8a"} err="failed to get container status \"ac691669ea7a5766986b91d76178c6feae5635a7049a196d165a08a5a92c1c8a\": rpc error: code = NotFound desc = could not find container \"ac691669ea7a5766986b91d76178c6feae5635a7049a196d165a08a5a92c1c8a\": container with ID starting with ac691669ea7a5766986b91d76178c6feae5635a7049a196d165a08a5a92c1c8a not found: ID does not exist" Sep 10 13:20:50.951429 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.951429 2560 scope.go:117] "RemoveContainer" containerID="3b987da2b6daccdf548c60335fb33085cf2d3d58027b5562894c64b117b0f610" Sep 10 13:20:50.951689 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.951668 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b987da2b6daccdf548c60335fb33085cf2d3d58027b5562894c64b117b0f610"} err="failed to get container status \"3b987da2b6daccdf548c60335fb33085cf2d3d58027b5562894c64b117b0f610\": rpc error: code = NotFound desc = could not find container \"3b987da2b6daccdf548c60335fb33085cf2d3d58027b5562894c64b117b0f610\": container with ID starting with 3b987da2b6daccdf548c60335fb33085cf2d3d58027b5562894c64b117b0f610 not found: ID does not exist" Sep 10 13:20:50.951728 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.951703 2560 scope.go:117] "RemoveContainer" containerID="1d6a5cbf688a0e4d0d9bd3c2538e5ebd4d830b54a409fd038a41a5f0ab92fecf" Sep 10 13:20:50.951972 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.951953 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d6a5cbf688a0e4d0d9bd3c2538e5ebd4d830b54a409fd038a41a5f0ab92fecf"} err="failed to get container status \"1d6a5cbf688a0e4d0d9bd3c2538e5ebd4d830b54a409fd038a41a5f0ab92fecf\": rpc error: code = NotFound desc = could not find container \"1d6a5cbf688a0e4d0d9bd3c2538e5ebd4d830b54a409fd038a41a5f0ab92fecf\": container with ID starting with 1d6a5cbf688a0e4d0d9bd3c2538e5ebd4d830b54a409fd038a41a5f0ab92fecf not found: ID does not exist" Sep 10 13:20:50.951972 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.951972 2560 scope.go:117] "RemoveContainer" containerID="d612f51b0b053f9abd930117864ad6f708f5fad939202d5c0974cf73a1b2b2c9" Sep 10 13:20:50.952201 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.952185 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d612f51b0b053f9abd930117864ad6f708f5fad939202d5c0974cf73a1b2b2c9"} err="failed to get container status \"d612f51b0b053f9abd930117864ad6f708f5fad939202d5c0974cf73a1b2b2c9\": rpc error: code = NotFound desc = could not find container \"d612f51b0b053f9abd930117864ad6f708f5fad939202d5c0974cf73a1b2b2c9\": container with ID starting with d612f51b0b053f9abd930117864ad6f708f5fad939202d5c0974cf73a1b2b2c9 not found: ID does not exist" Sep 10 13:20:50.952248 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.952201 2560 scope.go:117] "RemoveContainer" containerID="3fd215ba1d9dd060316b5b3ad36f8a035bc01f4603e770d22b85639c5eb8aed9" Sep 10 13:20:50.952427 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.952392 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fd215ba1d9dd060316b5b3ad36f8a035bc01f4603e770d22b85639c5eb8aed9"} err="failed to get container status \"3fd215ba1d9dd060316b5b3ad36f8a035bc01f4603e770d22b85639c5eb8aed9\": rpc error: code = NotFound desc = could not find container \"3fd215ba1d9dd060316b5b3ad36f8a035bc01f4603e770d22b85639c5eb8aed9\": container with ID starting with 3fd215ba1d9dd060316b5b3ad36f8a035bc01f4603e770d22b85639c5eb8aed9 not found: ID does not exist" Sep 10 13:20:50.952427 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.952426 2560 scope.go:117] "RemoveContainer" containerID="cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3" Sep 10 13:20:50.952663 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.952648 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3"} err="failed to get container status \"cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3\": rpc error: code = NotFound desc = could not find container \"cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3\": container with ID starting with cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3 not found: ID does not exist" Sep 10 13:20:50.952663 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.952663 2560 scope.go:117] "RemoveContainer" containerID="f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230" Sep 10 13:20:50.952921 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.952902 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230"} err="failed to get container status \"f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230\": rpc error: code = NotFound desc = could not find container \"f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230\": container with ID starting with f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230 not found: ID does not exist" Sep 10 13:20:50.952974 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.952922 2560 scope.go:117] "RemoveContainer" containerID="f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3" Sep 10 13:20:50.953264 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.953249 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3"} err="failed to get container status \"f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3\": rpc error: code = NotFound desc = could not find container \"f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3\": container with ID starting with f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3 not found: ID does not exist" Sep 10 13:20:50.953307 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.953264 2560 scope.go:117] "RemoveContainer" containerID="ac691669ea7a5766986b91d76178c6feae5635a7049a196d165a08a5a92c1c8a" Sep 10 13:20:50.953627 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.953606 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac691669ea7a5766986b91d76178c6feae5635a7049a196d165a08a5a92c1c8a"} err="failed to get container status \"ac691669ea7a5766986b91d76178c6feae5635a7049a196d165a08a5a92c1c8a\": rpc error: code = NotFound desc = could not find container \"ac691669ea7a5766986b91d76178c6feae5635a7049a196d165a08a5a92c1c8a\": container with ID starting with ac691669ea7a5766986b91d76178c6feae5635a7049a196d165a08a5a92c1c8a not found: ID does not exist" Sep 10 13:20:50.953693 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.953629 2560 scope.go:117] "RemoveContainer" containerID="3b987da2b6daccdf548c60335fb33085cf2d3d58027b5562894c64b117b0f610" Sep 10 13:20:50.953889 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.953872 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b987da2b6daccdf548c60335fb33085cf2d3d58027b5562894c64b117b0f610"} err="failed to get container status \"3b987da2b6daccdf548c60335fb33085cf2d3d58027b5562894c64b117b0f610\": rpc error: code = NotFound desc = could not find container \"3b987da2b6daccdf548c60335fb33085cf2d3d58027b5562894c64b117b0f610\": container with ID starting with 3b987da2b6daccdf548c60335fb33085cf2d3d58027b5562894c64b117b0f610 not found: ID does not exist" Sep 10 13:20:50.953947 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.953891 2560 scope.go:117] "RemoveContainer" containerID="1d6a5cbf688a0e4d0d9bd3c2538e5ebd4d830b54a409fd038a41a5f0ab92fecf" Sep 10 13:20:50.954147 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.954129 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d6a5cbf688a0e4d0d9bd3c2538e5ebd4d830b54a409fd038a41a5f0ab92fecf"} err="failed to get container status \"1d6a5cbf688a0e4d0d9bd3c2538e5ebd4d830b54a409fd038a41a5f0ab92fecf\": rpc error: code = NotFound desc = could not find container \"1d6a5cbf688a0e4d0d9bd3c2538e5ebd4d830b54a409fd038a41a5f0ab92fecf\": container with ID starting with 1d6a5cbf688a0e4d0d9bd3c2538e5ebd4d830b54a409fd038a41a5f0ab92fecf not found: ID does not exist" Sep 10 13:20:50.954147 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.954146 2560 scope.go:117] "RemoveContainer" containerID="d612f51b0b053f9abd930117864ad6f708f5fad939202d5c0974cf73a1b2b2c9" Sep 10 13:20:50.954436 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.954413 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d612f51b0b053f9abd930117864ad6f708f5fad939202d5c0974cf73a1b2b2c9"} err="failed to get container status \"d612f51b0b053f9abd930117864ad6f708f5fad939202d5c0974cf73a1b2b2c9\": rpc error: code = NotFound desc = could not find container \"d612f51b0b053f9abd930117864ad6f708f5fad939202d5c0974cf73a1b2b2c9\": container with ID starting with d612f51b0b053f9abd930117864ad6f708f5fad939202d5c0974cf73a1b2b2c9 not found: ID does not exist" Sep 10 13:20:50.954492 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.954439 2560 scope.go:117] "RemoveContainer" containerID="3fd215ba1d9dd060316b5b3ad36f8a035bc01f4603e770d22b85639c5eb8aed9" Sep 10 13:20:50.954699 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.954681 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fd215ba1d9dd060316b5b3ad36f8a035bc01f4603e770d22b85639c5eb8aed9"} err="failed to get container status \"3fd215ba1d9dd060316b5b3ad36f8a035bc01f4603e770d22b85639c5eb8aed9\": rpc error: code = NotFound desc = could not find container \"3fd215ba1d9dd060316b5b3ad36f8a035bc01f4603e770d22b85639c5eb8aed9\": container with ID starting with 3fd215ba1d9dd060316b5b3ad36f8a035bc01f4603e770d22b85639c5eb8aed9 not found: ID does not exist" Sep 10 13:20:50.954747 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.954699 2560 scope.go:117] "RemoveContainer" containerID="cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3" Sep 10 13:20:50.954958 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.954936 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3"} err="failed to get container status \"cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3\": rpc error: code = NotFound desc = could not find container \"cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3\": container with ID starting with cdb517dee4c89d9796ac416c827b285ac3e031215a6565c8e0b0d925dfc05ea3 not found: ID does not exist" Sep 10 13:20:50.955008 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.954960 2560 scope.go:117] "RemoveContainer" containerID="f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230" Sep 10 13:20:50.955203 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.955183 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230"} err="failed to get container status \"f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230\": rpc error: code = NotFound desc = could not find container \"f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230\": container with ID starting with f1420576a6f20c49541a2b20583847c2001e7c478cbbed6f2980967d2936b230 not found: ID does not exist" Sep 10 13:20:50.955203 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.955201 2560 scope.go:117] "RemoveContainer" containerID="f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3" Sep 10 13:20:50.955459 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:50.955438 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3"} err="failed to get container status \"f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3\": rpc error: code = NotFound desc = could not find container \"f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3\": container with ID starting with f37d11925b19effdfda8e7e653469bcf973e7ccbe2b9880aac5cca5d250c13a3 not found: ID does not exist" Sep 10 13:20:51.888054 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:51.888021 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" event={"ID":"edcc06ac-ef52-48ca-b6db-7c889342d723","Type":"ContainerStarted","Data":"0353899a4f2188ad349de8286c1126dcff877e82c457f67d7c9c9efbc2a4a091"} Sep 10 13:20:51.888467 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:51.888135 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" event={"ID":"edcc06ac-ef52-48ca-b6db-7c889342d723","Type":"ContainerStarted","Data":"bc77d52520f4803547561743fe9668c2c51a63a1f78e12dc198010b43bfb40d0"} Sep 10 13:20:51.888467 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:51.888169 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" event={"ID":"edcc06ac-ef52-48ca-b6db-7c889342d723","Type":"ContainerStarted","Data":"2d36476fd371125fb8309cb7d401af3ae5078f582e5ccb8219d436834be5f2a6"} Sep 10 13:20:51.888467 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:51.888197 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" event={"ID":"edcc06ac-ef52-48ca-b6db-7c889342d723","Type":"ContainerStarted","Data":"4447bb3a26c2d80f40d22085a0c6fd31fa2d3bd454780c29418ca2d04f0a4ba9"} Sep 10 13:20:52.556994 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:52.556970 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:20:52.557103 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:52.557026 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:20:52.557103 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:52.557025 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g64cr" Sep 10 13:20:52.557201 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:52.557114 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vrbmp" podUID="117705e7-b9e7-47ee-a474-d7f74afb23b6" Sep 10 13:20:52.557201 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:52.557184 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v2hld" podUID="faa8bfdc-9c79-4c12-8ec9-3cd8639d0793" Sep 10 13:20:52.557315 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:52.557224 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g64cr" podUID="3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597" Sep 10 13:20:52.560097 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:52.560075 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0db61932-33cb-4fcd-84e6-79282c285bbf" path="/var/lib/kubelet/pods/0db61932-33cb-4fcd-84e6-79282c285bbf/volumes" Sep 10 13:20:52.893280 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:52.893202 2560 generic.go:356] "Generic (PLEG): container finished" podID="cc3dddf2-363d-4c1d-824f-aaca292a88f0" containerID="22c17d4066315247d0a054d745acd31e3d2bfd47ffdcbbb5385b829b26956c89" exitCode=0 Sep 10 13:20:52.893280 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:52.893256 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-28xnt" event={"ID":"cc3dddf2-363d-4c1d-824f-aaca292a88f0","Type":"ContainerDied","Data":"22c17d4066315247d0a054d745acd31e3d2bfd47ffdcbbb5385b829b26956c89"} Sep 10 13:20:53.898332 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:53.898301 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" event={"ID":"edcc06ac-ef52-48ca-b6db-7c889342d723","Type":"ContainerStarted","Data":"526dfcb0d940be5f8ec8206baca3b7db14026cebee280ec115269cc245c80253"} Sep 10 13:20:54.557293 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:54.557261 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g64cr" Sep 10 13:20:54.557494 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:54.557261 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:20:54.557494 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:54.557271 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:20:54.557494 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:54.557414 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g64cr" podUID="3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597" Sep 10 13:20:54.557494 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:54.557482 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v2hld" podUID="faa8bfdc-9c79-4c12-8ec9-3cd8639d0793" Sep 10 13:20:54.557714 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:54.557564 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vrbmp" podUID="117705e7-b9e7-47ee-a474-d7f74afb23b6" Sep 10 13:20:55.502285 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:55.502249 2560 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 10 13:20:55.906243 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:55.906148 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" event={"ID":"edcc06ac-ef52-48ca-b6db-7c889342d723","Type":"ContainerStarted","Data":"29bd990a2c61cd3b6c564e5b1aba8477ae907b3919e434e5edf43375ca284944"} Sep 10 13:20:55.906560 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:55.906539 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:55.906686 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:55.906629 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:55.906686 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:55.906679 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:55.908922 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:55.908892 2560 generic.go:356] "Generic (PLEG): container finished" podID="cc3dddf2-363d-4c1d-824f-aaca292a88f0" containerID="761b5d669aa6082c5f2d24e86229ac20334c274cb2412344116dc8e0b0ab28d5" exitCode=0 Sep 10 13:20:55.909032 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:55.908940 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-28xnt" event={"ID":"cc3dddf2-363d-4c1d-824f-aaca292a88f0","Type":"ContainerDied","Data":"761b5d669aa6082c5f2d24e86229ac20334c274cb2412344116dc8e0b0ab28d5"} Sep 10 13:20:55.921916 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:55.921892 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:55.922031 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:55.921997 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:20:55.976606 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:55.976557 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" podStartSLOduration=5.976541546 podStartE2EDuration="5.976541546s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:20:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:20:55.948005638 +0000 UTC m=+165.842723055" watchObservedRunningTime="2026-09-10 13:20:55.976541546 +0000 UTC m=+165.871258998" Sep 10 13:20:56.557288 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:56.557257 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:20:56.557780 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:56.557259 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g64cr" Sep 10 13:20:56.557780 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:56.557363 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vrbmp" podUID="117705e7-b9e7-47ee-a474-d7f74afb23b6" Sep 10 13:20:56.557780 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:56.557459 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g64cr" podUID="3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597" Sep 10 13:20:56.557780 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:56.557259 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:20:56.557780 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:56.557535 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v2hld" podUID="faa8bfdc-9c79-4c12-8ec9-3cd8639d0793" Sep 10 13:20:58.557103 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:58.556867 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:20:58.557886 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:58.556868 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:20:58.557886 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:58.556868 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g64cr" Sep 10 13:20:58.557886 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:58.557229 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vrbmp" podUID="117705e7-b9e7-47ee-a474-d7f74afb23b6" Sep 10 13:20:58.557886 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:58.557358 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v2hld" podUID="faa8bfdc-9c79-4c12-8ec9-3cd8639d0793" Sep 10 13:20:58.557886 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:58.557445 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g64cr" podUID="3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597" Sep 10 13:20:58.686870 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:58.686838 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-g64cr"] Sep 10 13:20:58.690325 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:58.690258 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vrbmp"] Sep 10 13:20:58.690850 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:58.690829 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-v2hld"] Sep 10 13:20:58.779791 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:58.779719 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-jv5qh"] Sep 10 13:20:58.803326 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:58.803293 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jv5qh" Sep 10 13:20:58.805865 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:58.805844 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 10 13:20:58.806324 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:58.806301 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6m2vk\"" Sep 10 13:20:58.806576 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:58.806558 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 10 13:20:58.875686 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:58.875651 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7mzv\" (UniqueName: \"kubernetes.io/projected/75c33310-0b35-4c84-be13-75a0d65d95e9-kube-api-access-t7mzv\") pod \"node-resolver-jv5qh\" (UID: \"75c33310-0b35-4c84-be13-75a0d65d95e9\") " pod="openshift-dns/node-resolver-jv5qh" Sep 10 13:20:58.875861 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:58.875700 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/75c33310-0b35-4c84-be13-75a0d65d95e9-hosts-file\") pod \"node-resolver-jv5qh\" (UID: \"75c33310-0b35-4c84-be13-75a0d65d95e9\") " pod="openshift-dns/node-resolver-jv5qh" Sep 10 13:20:58.875861 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:58.875835 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75c33310-0b35-4c84-be13-75a0d65d95e9-tmp-dir\") pod \"node-resolver-jv5qh\" (UID: \"75c33310-0b35-4c84-be13-75a0d65d95e9\") " pod="openshift-dns/node-resolver-jv5qh" Sep 10 13:20:58.915480 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:58.915450 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:20:58.915642 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:58.915451 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:20:58.915642 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:58.915561 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v2hld" podUID="faa8bfdc-9c79-4c12-8ec9-3cd8639d0793" Sep 10 13:20:58.915735 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:58.915641 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vrbmp" podUID="117705e7-b9e7-47ee-a474-d7f74afb23b6" Sep 10 13:20:58.915735 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:58.915451 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g64cr" Sep 10 13:20:58.915735 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:58.915717 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g64cr" podUID="3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597" Sep 10 13:20:58.976432 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:58.976382 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/75c33310-0b35-4c84-be13-75a0d65d95e9-hosts-file\") pod \"node-resolver-jv5qh\" (UID: \"75c33310-0b35-4c84-be13-75a0d65d95e9\") " pod="openshift-dns/node-resolver-jv5qh" Sep 10 13:20:58.976591 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:58.976464 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/117705e7-b9e7-47ee-a474-d7f74afb23b6-metrics-certs\") pod \"network-metrics-daemon-vrbmp\" (UID: \"117705e7-b9e7-47ee-a474-d7f74afb23b6\") " pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:20:58.976591 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:58.976491 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75c33310-0b35-4c84-be13-75a0d65d95e9-tmp-dir\") pod \"node-resolver-jv5qh\" (UID: \"75c33310-0b35-4c84-be13-75a0d65d95e9\") " pod="openshift-dns/node-resolver-jv5qh" Sep 10 13:20:58.976591 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:58.976537 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7mzv\" (UniqueName: \"kubernetes.io/projected/75c33310-0b35-4c84-be13-75a0d65d95e9-kube-api-access-t7mzv\") pod \"node-resolver-jv5qh\" (UID: \"75c33310-0b35-4c84-be13-75a0d65d95e9\") " pod="openshift-dns/node-resolver-jv5qh" Sep 10 13:20:58.976591 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:58.976549 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/75c33310-0b35-4c84-be13-75a0d65d95e9-hosts-file\") pod \"node-resolver-jv5qh\" (UID: \"75c33310-0b35-4c84-be13-75a0d65d95e9\") " pod="openshift-dns/node-resolver-jv5qh" Sep 10 13:20:58.976755 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:58.976660 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 13:20:58.976755 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:58.976736 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/117705e7-b9e7-47ee-a474-d7f74afb23b6-metrics-certs podName:117705e7-b9e7-47ee-a474-d7f74afb23b6 nodeName:}" failed. No retries permitted until 2026-09-10 13:21:30.976714876 +0000 UTC m=+200.871432276 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/117705e7-b9e7-47ee-a474-d7f74afb23b6-metrics-certs") pod "network-metrics-daemon-vrbmp" (UID: "117705e7-b9e7-47ee-a474-d7f74afb23b6") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 13:20:58.976881 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:58.976861 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75c33310-0b35-4c84-be13-75a0d65d95e9-tmp-dir\") pod \"node-resolver-jv5qh\" (UID: \"75c33310-0b35-4c84-be13-75a0d65d95e9\") " pod="openshift-dns/node-resolver-jv5qh" Sep 10 13:20:58.986766 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:58.986745 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7mzv\" (UniqueName: \"kubernetes.io/projected/75c33310-0b35-4c84-be13-75a0d65d95e9-kube-api-access-t7mzv\") pod \"node-resolver-jv5qh\" (UID: \"75c33310-0b35-4c84-be13-75a0d65d95e9\") " pod="openshift-dns/node-resolver-jv5qh" Sep 10 13:20:59.077296 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:59.077186 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28v55\" (UniqueName: \"kubernetes.io/projected/3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597-kube-api-access-28v55\") pod \"network-check-target-g64cr\" (UID: \"3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597\") " pod="openshift-network-diagnostics/network-check-target-g64cr" Sep 10 13:20:59.077475 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:59.077385 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 13:20:59.077475 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:59.077425 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 13:20:59.077475 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:59.077438 2560 projected.go:196] Error preparing data for projected volume kube-api-access-28v55 for pod openshift-network-diagnostics/network-check-target-g64cr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 13:20:59.077639 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:59.077503 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597-kube-api-access-28v55 podName:3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597 nodeName:}" failed. No retries permitted until 2026-09-10 13:21:31.077484002 +0000 UTC m=+200.972201401 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-28v55" (UniqueName: "kubernetes.io/projected/3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597-kube-api-access-28v55") pod "network-check-target-g64cr" (UID: "3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 13:20:59.114522 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:59.114491 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jv5qh" Sep 10 13:20:59.278327 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:20:59.278280 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/faa8bfdc-9c79-4c12-8ec9-3cd8639d0793-original-pull-secret\") pod \"global-pull-secret-syncer-v2hld\" (UID: \"faa8bfdc-9c79-4c12-8ec9-3cd8639d0793\") " pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:20:59.278501 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:59.278430 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 13:20:59.278501 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:20:59.278492 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faa8bfdc-9c79-4c12-8ec9-3cd8639d0793-original-pull-secret podName:faa8bfdc-9c79-4c12-8ec9-3cd8639d0793 nodeName:}" failed. No retries permitted until 2026-09-10 13:21:31.27847726 +0000 UTC m=+201.173194676 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/faa8bfdc-9c79-4c12-8ec9-3cd8639d0793-original-pull-secret") pod "global-pull-secret-syncer-v2hld" (UID: "faa8bfdc-9c79-4c12-8ec9-3cd8639d0793") : object "kube-system"/"original-pull-secret" not registered Sep 10 13:21:00.503269 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:00.503230 2560 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 10 13:21:00.544131 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:21:00.544100 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75c33310_0b35_4c84_be13_75a0d65d95e9.slice/crio-bf93cf36b088a399a082e3816f47f84529e2cbbb584bf6731364c85bda6a7458 WatchSource:0}: Error finding container bf93cf36b088a399a082e3816f47f84529e2cbbb584bf6731364c85bda6a7458: Status 404 returned error can't find the container with id bf93cf36b088a399a082e3816f47f84529e2cbbb584bf6731364c85bda6a7458 Sep 10 13:21:00.557884 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:00.557853 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:21:00.558009 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:00.557984 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v2hld" podUID="faa8bfdc-9c79-4c12-8ec9-3cd8639d0793" Sep 10 13:21:00.558062 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:00.558030 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g64cr" Sep 10 13:21:00.558138 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:00.558121 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g64cr" podUID="3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597" Sep 10 13:21:00.558176 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:00.558151 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:21:00.558243 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:00.558228 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vrbmp" podUID="117705e7-b9e7-47ee-a474-d7f74afb23b6" Sep 10 13:21:00.924074 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:00.923855 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-28xnt" event={"ID":"cc3dddf2-363d-4c1d-824f-aaca292a88f0","Type":"ContainerStarted","Data":"ef9cf2dcbf7fe040979a23b9a3283418521613d14a016b9eec0db6fa34c595e1"} Sep 10 13:21:00.925606 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:00.925583 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jv5qh" event={"ID":"75c33310-0b35-4c84-be13-75a0d65d95e9","Type":"ContainerStarted","Data":"6889edbbf0ed082b913ef6ff24693836ea1a3d529dd371fd32c3c6d567143adb"} Sep 10 13:21:00.925702 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:00.925626 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jv5qh" event={"ID":"75c33310-0b35-4c84-be13-75a0d65d95e9","Type":"ContainerStarted","Data":"bf93cf36b088a399a082e3816f47f84529e2cbbb584bf6731364c85bda6a7458"} Sep 10 13:21:00.942641 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:00.942319 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-jv5qh" podStartSLOduration=2.9423013449999997 podStartE2EDuration="2.942301345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:20:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:21:00.941973769 +0000 UTC m=+170.836691396" watchObservedRunningTime="2026-09-10 13:21:00.942301345 +0000 UTC m=+170.837018767" Sep 10 13:21:01.930823 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:01.930790 2560 generic.go:356] "Generic (PLEG): container finished" podID="cc3dddf2-363d-4c1d-824f-aaca292a88f0" containerID="ef9cf2dcbf7fe040979a23b9a3283418521613d14a016b9eec0db6fa34c595e1" exitCode=0 Sep 10 13:21:01.931281 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:01.930849 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-28xnt" event={"ID":"cc3dddf2-363d-4c1d-824f-aaca292a88f0","Type":"ContainerDied","Data":"ef9cf2dcbf7fe040979a23b9a3283418521613d14a016b9eec0db6fa34c595e1"} Sep 10 13:21:02.557487 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:02.557440 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:21:02.557671 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:02.557440 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g64cr" Sep 10 13:21:02.557671 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:02.557576 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:21:02.557671 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:02.557569 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vrbmp" podUID="117705e7-b9e7-47ee-a474-d7f74afb23b6" Sep 10 13:21:02.557671 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:02.557657 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g64cr" podUID="3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597" Sep 10 13:21:02.557818 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:02.557747 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v2hld" podUID="faa8bfdc-9c79-4c12-8ec9-3cd8639d0793" Sep 10 13:21:02.936189 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:02.936152 2560 generic.go:356] "Generic (PLEG): container finished" podID="cc3dddf2-363d-4c1d-824f-aaca292a88f0" containerID="9928b265823e2cb6277da370b0b5202486b1183b72066fb20afa0ca1be6a1b87" exitCode=0 Sep 10 13:21:02.936189 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:02.936191 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-28xnt" event={"ID":"cc3dddf2-363d-4c1d-824f-aaca292a88f0","Type":"ContainerDied","Data":"9928b265823e2cb6277da370b0b5202486b1183b72066fb20afa0ca1be6a1b87"} Sep 10 13:21:03.941829 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:03.941790 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-28xnt" event={"ID":"cc3dddf2-363d-4c1d-824f-aaca292a88f0","Type":"ContainerStarted","Data":"6c046d074fc05af87feba32dd9c304ace7e66e96a80956ca97d0408acb52a545"} Sep 10 13:21:03.963241 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:03.963193 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-28xnt" podStartSLOduration=8.617896934000001 podStartE2EDuration="37.96317786s" totalImagesPullingTime="29.345280926s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:20:26 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:21:03.962896643 +0000 UTC m=+173.857614067" watchObservedRunningTime="2026-09-10 13:21:03.96317786 +0000 UTC m=+173.857895275" Sep 10 13:21:04.557376 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:04.557338 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:21:04.557685 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:04.557338 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:21:04.557685 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:04.557348 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g64cr" Sep 10 13:21:04.557685 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:04.557499 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vrbmp" podUID="117705e7-b9e7-47ee-a474-d7f74afb23b6" Sep 10 13:21:04.557685 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:04.557615 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g64cr" podUID="3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597" Sep 10 13:21:04.557685 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:04.557669 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v2hld" podUID="faa8bfdc-9c79-4c12-8ec9-3cd8639d0793" Sep 10 13:21:06.556619 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:06.556579 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g64cr" Sep 10 13:21:06.557068 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:06.556586 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:21:06.557068 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:06.556585 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:21:06.563139 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:06.563054 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 10 13:21:06.563461 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:06.563444 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 10 13:21:06.564162 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:06.564144 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-67hjc\"" Sep 10 13:21:06.564234 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:06.564183 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 13:21:06.564472 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:06.564454 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4sjdq\"" Sep 10 13:21:06.564563 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:06.564495 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 13:21:09.740933 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.740901 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-118.ec2.internal" event="NodeReady" Sep 10 13:21:09.780847 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.780811 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8k875"] Sep 10 13:21:09.783007 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.782986 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8k875" Sep 10 13:21:09.786939 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.786901 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-mnksz"] Sep 10 13:21:09.787444 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.787426 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Sep 10 13:21:09.787567 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.787448 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 10 13:21:09.788655 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.788634 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-mnksz" Sep 10 13:21:09.789549 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.788635 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-x85fh\"" Sep 10 13:21:09.789549 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.789503 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 10 13:21:09.789873 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.789853 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-482xw"] Sep 10 13:21:09.790875 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.790344 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Sep 10 13:21:09.791730 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.791456 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qkdhm"] Sep 10 13:21:09.791730 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.791564 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-482xw" Sep 10 13:21:09.793351 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.793335 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7d5587b5d-tzd9q"] Sep 10 13:21:09.793494 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.793476 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qkdhm" Sep 10 13:21:09.794987 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.794972 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-gtqlh"] Sep 10 13:21:09.795117 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.795082 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" Sep 10 13:21:09.796482 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.796463 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4hmcg"] Sep 10 13:21:09.796576 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.796555 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-gtqlh" Sep 10 13:21:09.797951 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.797933 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5787c86768-n9xzb"] Sep 10 13:21:09.798046 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.798026 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4hmcg" Sep 10 13:21:09.799512 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.799497 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-bdnkb"] Sep 10 13:21:09.799622 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.799607 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5787c86768-n9xzb" Sep 10 13:21:09.800933 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.800919 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-9zr8q"] Sep 10 13:21:09.801031 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.801018 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bdnkb" Sep 10 13:21:09.801665 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.801645 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 10 13:21:09.801862 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.801842 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Sep 10 13:21:09.801862 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.801861 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Sep 10 13:21:09.802299 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.802280 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Sep 10 13:21:09.802446 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.802428 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-vm8ms\"" Sep 10 13:21:09.802831 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.802815 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-qw7g8"] Sep 10 13:21:09.802907 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.802829 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Sep 10 13:21:09.802907 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.802866 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-9zr8q" Sep 10 13:21:09.803010 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.802819 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Sep 10 13:21:09.803217 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.803196 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Sep 10 13:21:09.803299 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.803235 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 10 13:21:09.803299 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.803235 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 10 13:21:09.805851 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.805658 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Sep 10 13:21:09.806193 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.806171 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-l9n98\"" Sep 10 13:21:09.806585 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.806560 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 10 13:21:09.806759 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.806739 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-qw7g8" Sep 10 13:21:09.808645 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.808445 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Sep 10 13:21:09.808645 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.808499 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Sep 10 13:21:09.808645 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.808507 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-j8q7v\"" Sep 10 13:21:09.808645 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.808513 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 10 13:21:09.808645 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.808502 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-gwdml\"" Sep 10 13:21:09.808645 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.808634 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Sep 10 13:21:09.809097 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.808654 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 10 13:21:09.809097 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.808657 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Sep 10 13:21:09.809097 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.808701 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Sep 10 13:21:09.809097 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.808707 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Sep 10 13:21:09.809097 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.808822 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Sep 10 13:21:09.809097 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.808836 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-cpg6c\"" Sep 10 13:21:09.809097 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.808962 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Sep 10 13:21:09.809097 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.809003 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Sep 10 13:21:09.809097 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.809039 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gf67b\"" Sep 10 13:21:09.809515 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.809196 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Sep 10 13:21:09.809515 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.809217 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Sep 10 13:21:09.809907 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.809879 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Sep 10 13:21:09.810099 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.810084 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 10 13:21:09.810287 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.810238 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-jnx5s\"" Sep 10 13:21:09.810460 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.810389 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Sep 10 13:21:09.810532 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.810502 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-vvl5k\"" Sep 10 13:21:09.810588 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.810530 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Sep 10 13:21:09.810641 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.810609 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-q6z4v\"" Sep 10 13:21:09.810691 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.810649 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Sep 10 13:21:09.810691 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.810670 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Sep 10 13:21:09.815343 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.815312 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Sep 10 13:21:09.819293 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.819277 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-qw7g8"] Sep 10 13:21:09.820039 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.820022 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qkdhm"] Sep 10 13:21:09.824055 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.824037 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-bdnkb"] Sep 10 13:21:09.825125 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.825104 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-gtqlh"] Sep 10 13:21:09.825886 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.825870 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-482xw"] Sep 10 13:21:09.826756 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.826735 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-9zr8q"] Sep 10 13:21:09.827785 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.827761 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Sep 10 13:21:09.827927 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.827910 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Sep 10 13:21:09.828064 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.827957 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2tqpb"] Sep 10 13:21:09.830273 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.830255 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8k875"] Sep 10 13:21:09.830384 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.830365 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2tqpb" Sep 10 13:21:09.842055 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.842032 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-zgs9h"] Sep 10 13:21:09.844018 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.843997 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zgs9h" Sep 10 13:21:09.847068 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.847042 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 10 13:21:09.847170 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.847137 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 10 13:21:09.848047 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.848031 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-qg65m\"" Sep 10 13:21:09.848612 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.848581 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Sep 10 13:21:09.849115 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.849096 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 10 13:21:09.852444 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.852427 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 10 13:21:09.853141 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.852456 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7k2sl\"" Sep 10 13:21:09.853141 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.852986 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 10 13:21:09.853767 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.853750 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 10 13:21:09.854354 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.854172 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-nzsjz\"" Sep 10 13:21:09.854543 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.854457 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2tqpb"] Sep 10 13:21:09.854760 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.854740 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4hmcg"] Sep 10 13:21:09.861543 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.861521 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-mnksz"] Sep 10 13:21:09.862179 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.862160 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5787c86768-n9xzb"] Sep 10 13:21:09.862773 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.862753 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d5587b5d-tzd9q"] Sep 10 13:21:09.867557 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.867539 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zgs9h"] Sep 10 13:21:09.947237 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.947202 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-service-ca-bundle\") pod \"router-default-5787c86768-n9xzb\" (UID: \"b19de8c5-85f7-4f5e-9011-8f38d0b80a69\") " pod="openshift-ingress/router-default-5787c86768-n9xzb" Sep 10 13:21:09.947424 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.947258 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/74a231ea-0f6d-4d1a-9e98-b54fa96a73bb-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-qkdhm\" (UID: \"74a231ea-0f6d-4d1a-9e98-b54fa96a73bb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qkdhm" Sep 10 13:21:09.947424 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.947280 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9c97ef99-3ac3-4334-aba7-f9de4a6cccf6-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-4hmcg\" (UID: \"9c97ef99-3ac3-4334-aba7-f9de4a6cccf6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4hmcg" Sep 10 13:21:09.947424 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.947297 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7fe46372-65c2-451c-9fba-11399f8c753d-config-volume\") pod \"dns-default-2tqpb\" (UID: \"7fe46372-65c2-451c-9fba-11399f8c753d\") " pod="openshift-dns/dns-default-2tqpb" Sep 10 13:21:09.947424 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.947314 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/227a2714-52ab-48f6-904b-9e997ed5f3a3-cert\") pod \"ingress-canary-zgs9h\" (UID: \"227a2714-52ab-48f6-904b-9e997ed5f3a3\") " pod="openshift-ingress-canary/ingress-canary-zgs9h" Sep 10 13:21:09.947424 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.947368 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4fe9733a-2777-4507-af4b-29f330350747-serving-cert\") pod \"insights-operator-76bcd6ccf5-mnksz\" (UID: \"4fe9733a-2777-4507-af4b-29f330350747\") " pod="openshift-insights/insights-operator-76bcd6ccf5-mnksz" Sep 10 13:21:09.947424 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.947418 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c58mw\" (UniqueName: \"kubernetes.io/projected/52fceedd-00f2-4943-ae47-dec51216d8e4-kube-api-access-c58mw\") pod \"console-operator-85b454874b-gtqlh\" (UID: \"52fceedd-00f2-4943-ae47-dec51216d8e4\") " pod="openshift-console-operator/console-operator-85b454874b-gtqlh" Sep 10 13:21:09.947652 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.947446 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3ef3618e-e54b-46bf-8347-69b8e72720fd-bound-sa-token\") pod \"image-registry-7d5587b5d-tzd9q\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" Sep 10 13:21:09.947652 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.947462 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-metrics-certs\") pod \"router-default-5787c86768-n9xzb\" (UID: \"b19de8c5-85f7-4f5e-9011-8f38d0b80a69\") " pod="openshift-ingress/router-default-5787c86768-n9xzb" Sep 10 13:21:09.947652 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.947479 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9c97ef99-3ac3-4334-aba7-f9de4a6cccf6-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-4hmcg\" (UID: \"9c97ef99-3ac3-4334-aba7-f9de4a6cccf6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4hmcg" Sep 10 13:21:09.947652 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.947498 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3ef3618e-e54b-46bf-8347-69b8e72720fd-ca-trust-extracted\") pod \"image-registry-7d5587b5d-tzd9q\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" Sep 10 13:21:09.947652 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.947521 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlvjh\" (UniqueName: \"kubernetes.io/projected/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-kube-api-access-wlvjh\") pod \"router-default-5787c86768-n9xzb\" (UID: \"b19de8c5-85f7-4f5e-9011-8f38d0b80a69\") " pod="openshift-ingress/router-default-5787c86768-n9xzb" Sep 10 13:21:09.947652 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.947538 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzrsc\" (UniqueName: \"kubernetes.io/projected/d763e40a-94ec-4167-8962-c355da8574d0-kube-api-access-nzrsc\") pod \"service-ca-operator-859f94c4b-bdnkb\" (UID: \"d763e40a-94ec-4167-8962-c355da8574d0\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bdnkb" Sep 10 13:21:09.947652 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.947559 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/52fceedd-00f2-4943-ae47-dec51216d8e4-serving-cert\") pod \"console-operator-85b454874b-gtqlh\" (UID: \"52fceedd-00f2-4943-ae47-dec51216d8e4\") " pod="openshift-console-operator/console-operator-85b454874b-gtqlh" Sep 10 13:21:09.947652 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.947577 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hp6l\" (UniqueName: \"kubernetes.io/projected/04022e0a-02cd-4c63-a372-6b9f600c80cd-kube-api-access-9hp6l\") pod \"network-check-source-685456655f-9zr8q\" (UID: \"04022e0a-02cd-4c63-a372-6b9f600c80cd\") " pod="openshift-network-diagnostics/network-check-source-685456655f-9zr8q" Sep 10 13:21:09.947652 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.947603 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4fe9733a-2777-4507-af4b-29f330350747-snapshots\") pod \"insights-operator-76bcd6ccf5-mnksz\" (UID: \"4fe9733a-2777-4507-af4b-29f330350747\") " pod="openshift-insights/insights-operator-76bcd6ccf5-mnksz" Sep 10 13:21:09.947652 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.947624 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7fe46372-65c2-451c-9fba-11399f8c753d-metrics-tls\") pod \"dns-default-2tqpb\" (UID: \"7fe46372-65c2-451c-9fba-11399f8c753d\") " pod="openshift-dns/dns-default-2tqpb" Sep 10 13:21:09.947652 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.947643 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3ef3618e-e54b-46bf-8347-69b8e72720fd-registry-tls\") pod \"image-registry-7d5587b5d-tzd9q\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" Sep 10 13:21:09.948066 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.947666 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4fe9733a-2777-4507-af4b-29f330350747-tmp\") pod \"insights-operator-76bcd6ccf5-mnksz\" (UID: \"4fe9733a-2777-4507-af4b-29f330350747\") " pod="openshift-insights/insights-operator-76bcd6ccf5-mnksz" Sep 10 13:21:09.948066 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.947710 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/4a051c0c-418c-439b-94e1-181c0a84e86b-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-8k875\" (UID: \"4a051c0c-418c-439b-94e1-181c0a84e86b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8k875" Sep 10 13:21:09.948066 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.947776 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fe9733a-2777-4507-af4b-29f330350747-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-mnksz\" (UID: \"4fe9733a-2777-4507-af4b-29f330350747\") " pod="openshift-insights/insights-operator-76bcd6ccf5-mnksz" Sep 10 13:21:09.948066 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.947832 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3ef3618e-e54b-46bf-8347-69b8e72720fd-registry-certificates\") pod \"image-registry-7d5587b5d-tzd9q\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" Sep 10 13:21:09.948066 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.947850 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdfg8\" (UniqueName: \"kubernetes.io/projected/4fe9733a-2777-4507-af4b-29f330350747-kube-api-access-tdfg8\") pod \"insights-operator-76bcd6ccf5-mnksz\" (UID: \"4fe9733a-2777-4507-af4b-29f330350747\") " pod="openshift-insights/insights-operator-76bcd6ccf5-mnksz" Sep 10 13:21:09.948066 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.947869 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-stats-auth\") pod \"router-default-5787c86768-n9xzb\" (UID: \"b19de8c5-85f7-4f5e-9011-8f38d0b80a69\") " pod="openshift-ingress/router-default-5787c86768-n9xzb" Sep 10 13:21:09.948066 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.947902 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjxrf\" (UniqueName: \"kubernetes.io/projected/74a231ea-0f6d-4d1a-9e98-b54fa96a73bb-kube-api-access-cjxrf\") pod \"cluster-samples-operator-7cdcfcf76c-qkdhm\" (UID: \"74a231ea-0f6d-4d1a-9e98-b54fa96a73bb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qkdhm" Sep 10 13:21:09.948066 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.947926 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e7fc744a-ed33-44a7-94f1-19bbd01b9c50-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-qw7g8\" (UID: \"e7fc744a-ed33-44a7-94f1-19bbd01b9c50\") " pod="openshift-network-console/networking-console-plugin-79977f9844-qw7g8" Sep 10 13:21:09.948066 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.947942 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3ef3618e-e54b-46bf-8347-69b8e72720fd-image-registry-private-configuration\") pod \"image-registry-7d5587b5d-tzd9q\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" Sep 10 13:21:09.948066 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.947966 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/52fceedd-00f2-4943-ae47-dec51216d8e4-config\") pod \"console-operator-85b454874b-gtqlh\" (UID: \"52fceedd-00f2-4943-ae47-dec51216d8e4\") " pod="openshift-console-operator/console-operator-85b454874b-gtqlh" Sep 10 13:21:09.948066 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.948014 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfld6\" (UniqueName: \"kubernetes.io/projected/9c97ef99-3ac3-4334-aba7-f9de4a6cccf6-kube-api-access-nfld6\") pod \"kube-storage-version-migrator-operator-8d7dd4675-4hmcg\" (UID: \"9c97ef99-3ac3-4334-aba7-f9de4a6cccf6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4hmcg" Sep 10 13:21:09.948066 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.948030 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e7fc744a-ed33-44a7-94f1-19bbd01b9c50-nginx-conf\") pod \"networking-console-plugin-79977f9844-qw7g8\" (UID: \"e7fc744a-ed33-44a7-94f1-19bbd01b9c50\") " pod="openshift-network-console/networking-console-plugin-79977f9844-qw7g8" Sep 10 13:21:09.948066 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.948045 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5djq\" (UniqueName: \"kubernetes.io/projected/227a2714-52ab-48f6-904b-9e997ed5f3a3-kube-api-access-l5djq\") pod \"ingress-canary-zgs9h\" (UID: \"227a2714-52ab-48f6-904b-9e997ed5f3a3\") " pod="openshift-ingress-canary/ingress-canary-zgs9h" Sep 10 13:21:09.948066 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.948058 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d763e40a-94ec-4167-8962-c355da8574d0-serving-cert\") pod \"service-ca-operator-859f94c4b-bdnkb\" (UID: \"d763e40a-94ec-4167-8962-c355da8574d0\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bdnkb" Sep 10 13:21:09.948649 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.948081 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnl9t\" (UniqueName: \"kubernetes.io/projected/4a051c0c-418c-439b-94e1-181c0a84e86b-kube-api-access-xnl9t\") pod \"cluster-monitoring-operator-6cdcc96965-8k875\" (UID: \"4a051c0c-418c-439b-94e1-181c0a84e86b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8k875" Sep 10 13:21:09.948649 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.948097 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fe9733a-2777-4507-af4b-29f330350747-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-mnksz\" (UID: \"4fe9733a-2777-4507-af4b-29f330350747\") " pod="openshift-insights/insights-operator-76bcd6ccf5-mnksz" Sep 10 13:21:09.948649 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.948127 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/52fceedd-00f2-4943-ae47-dec51216d8e4-trusted-ca\") pod \"console-operator-85b454874b-gtqlh\" (UID: \"52fceedd-00f2-4943-ae47-dec51216d8e4\") " pod="openshift-console-operator/console-operator-85b454874b-gtqlh" Sep 10 13:21:09.948649 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.948160 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-default-certificate\") pod \"router-default-5787c86768-n9xzb\" (UID: \"b19de8c5-85f7-4f5e-9011-8f38d0b80a69\") " pod="openshift-ingress/router-default-5787c86768-n9xzb" Sep 10 13:21:09.948649 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.948185 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d763e40a-94ec-4167-8962-c355da8574d0-config\") pod \"service-ca-operator-859f94c4b-bdnkb\" (UID: \"d763e40a-94ec-4167-8962-c355da8574d0\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bdnkb" Sep 10 13:21:09.948649 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.948209 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fe46372-65c2-451c-9fba-11399f8c753d-tmp-dir\") pod \"dns-default-2tqpb\" (UID: \"7fe46372-65c2-451c-9fba-11399f8c753d\") " pod="openshift-dns/dns-default-2tqpb" Sep 10 13:21:09.948649 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.948249 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4a051c0c-418c-439b-94e1-181c0a84e86b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-8k875\" (UID: \"4a051c0c-418c-439b-94e1-181c0a84e86b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8k875" Sep 10 13:21:09.948649 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.948284 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqqbs\" (UniqueName: \"kubernetes.io/projected/ce4cfb8d-701a-414f-99ac-5f73e57c81ab-kube-api-access-bqqbs\") pod \"volume-data-source-validator-695c895bd6-482xw\" (UID: \"ce4cfb8d-701a-414f-99ac-5f73e57c81ab\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-482xw" Sep 10 13:21:09.948649 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.948309 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbcrn\" (UniqueName: \"kubernetes.io/projected/3ef3618e-e54b-46bf-8347-69b8e72720fd-kube-api-access-tbcrn\") pod \"image-registry-7d5587b5d-tzd9q\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" Sep 10 13:21:09.948649 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.948335 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3ef3618e-e54b-46bf-8347-69b8e72720fd-trusted-ca\") pod \"image-registry-7d5587b5d-tzd9q\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" Sep 10 13:21:09.948649 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.948360 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkrf7\" (UniqueName: \"kubernetes.io/projected/7fe46372-65c2-451c-9fba-11399f8c753d-kube-api-access-dkrf7\") pod \"dns-default-2tqpb\" (UID: \"7fe46372-65c2-451c-9fba-11399f8c753d\") " pod="openshift-dns/dns-default-2tqpb" Sep 10 13:21:09.948649 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:09.948386 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3ef3618e-e54b-46bf-8347-69b8e72720fd-installation-pull-secrets\") pod \"image-registry-7d5587b5d-tzd9q\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" Sep 10 13:21:10.049318 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.049232 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fe9733a-2777-4507-af4b-29f330350747-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-mnksz\" (UID: \"4fe9733a-2777-4507-af4b-29f330350747\") " pod="openshift-insights/insights-operator-76bcd6ccf5-mnksz" Sep 10 13:21:10.049318 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.049266 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3ef3618e-e54b-46bf-8347-69b8e72720fd-registry-certificates\") pod \"image-registry-7d5587b5d-tzd9q\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" Sep 10 13:21:10.049318 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.049283 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdfg8\" (UniqueName: \"kubernetes.io/projected/4fe9733a-2777-4507-af4b-29f330350747-kube-api-access-tdfg8\") pod \"insights-operator-76bcd6ccf5-mnksz\" (UID: \"4fe9733a-2777-4507-af4b-29f330350747\") " pod="openshift-insights/insights-operator-76bcd6ccf5-mnksz" Sep 10 13:21:10.049642 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.049581 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-stats-auth\") pod \"router-default-5787c86768-n9xzb\" (UID: \"b19de8c5-85f7-4f5e-9011-8f38d0b80a69\") " pod="openshift-ingress/router-default-5787c86768-n9xzb" Sep 10 13:21:10.049642 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.049628 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjxrf\" (UniqueName: \"kubernetes.io/projected/74a231ea-0f6d-4d1a-9e98-b54fa96a73bb-kube-api-access-cjxrf\") pod \"cluster-samples-operator-7cdcfcf76c-qkdhm\" (UID: \"74a231ea-0f6d-4d1a-9e98-b54fa96a73bb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qkdhm" Sep 10 13:21:10.049744 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.049659 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e7fc744a-ed33-44a7-94f1-19bbd01b9c50-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-qw7g8\" (UID: \"e7fc744a-ed33-44a7-94f1-19bbd01b9c50\") " pod="openshift-network-console/networking-console-plugin-79977f9844-qw7g8" Sep 10 13:21:10.049744 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.049687 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3ef3618e-e54b-46bf-8347-69b8e72720fd-image-registry-private-configuration\") pod \"image-registry-7d5587b5d-tzd9q\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" Sep 10 13:21:10.049744 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.049704 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/52fceedd-00f2-4943-ae47-dec51216d8e4-config\") pod \"console-operator-85b454874b-gtqlh\" (UID: \"52fceedd-00f2-4943-ae47-dec51216d8e4\") " pod="openshift-console-operator/console-operator-85b454874b-gtqlh" Sep 10 13:21:10.049744 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.049725 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfld6\" (UniqueName: \"kubernetes.io/projected/9c97ef99-3ac3-4334-aba7-f9de4a6cccf6-kube-api-access-nfld6\") pod \"kube-storage-version-migrator-operator-8d7dd4675-4hmcg\" (UID: \"9c97ef99-3ac3-4334-aba7-f9de4a6cccf6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4hmcg" Sep 10 13:21:10.049973 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:10.049750 2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 13:21:10.049973 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.049754 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e7fc744a-ed33-44a7-94f1-19bbd01b9c50-nginx-conf\") pod \"networking-console-plugin-79977f9844-qw7g8\" (UID: \"e7fc744a-ed33-44a7-94f1-19bbd01b9c50\") " pod="openshift-network-console/networking-console-plugin-79977f9844-qw7g8" Sep 10 13:21:10.049973 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.049780 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5djq\" (UniqueName: \"kubernetes.io/projected/227a2714-52ab-48f6-904b-9e997ed5f3a3-kube-api-access-l5djq\") pod \"ingress-canary-zgs9h\" (UID: \"227a2714-52ab-48f6-904b-9e997ed5f3a3\") " pod="openshift-ingress-canary/ingress-canary-zgs9h" Sep 10 13:21:10.049973 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:10.049812 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7fc744a-ed33-44a7-94f1-19bbd01b9c50-networking-console-plugin-cert podName:e7fc744a-ed33-44a7-94f1-19bbd01b9c50 nodeName:}" failed. No retries permitted until 2026-09-10 13:21:10.549790395 +0000 UTC m=+180.444507796 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e7fc744a-ed33-44a7-94f1-19bbd01b9c50-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-qw7g8" (UID: "e7fc744a-ed33-44a7-94f1-19bbd01b9c50") : secret "networking-console-plugin-cert" not found Sep 10 13:21:10.049973 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.049832 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d763e40a-94ec-4167-8962-c355da8574d0-serving-cert\") pod \"service-ca-operator-859f94c4b-bdnkb\" (UID: \"d763e40a-94ec-4167-8962-c355da8574d0\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bdnkb" Sep 10 13:21:10.049973 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.049875 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnl9t\" (UniqueName: \"kubernetes.io/projected/4a051c0c-418c-439b-94e1-181c0a84e86b-kube-api-access-xnl9t\") pod \"cluster-monitoring-operator-6cdcc96965-8k875\" (UID: \"4a051c0c-418c-439b-94e1-181c0a84e86b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8k875" Sep 10 13:21:10.049973 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.049906 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fe9733a-2777-4507-af4b-29f330350747-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-mnksz\" (UID: \"4fe9733a-2777-4507-af4b-29f330350747\") " pod="openshift-insights/insights-operator-76bcd6ccf5-mnksz" Sep 10 13:21:10.049973 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.049931 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/52fceedd-00f2-4943-ae47-dec51216d8e4-trusted-ca\") pod \"console-operator-85b454874b-gtqlh\" (UID: \"52fceedd-00f2-4943-ae47-dec51216d8e4\") " pod="openshift-console-operator/console-operator-85b454874b-gtqlh" Sep 10 13:21:10.049973 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.049963 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-default-certificate\") pod \"router-default-5787c86768-n9xzb\" (UID: \"b19de8c5-85f7-4f5e-9011-8f38d0b80a69\") " pod="openshift-ingress/router-default-5787c86768-n9xzb" Sep 10 13:21:10.050459 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.049986 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d763e40a-94ec-4167-8962-c355da8574d0-config\") pod \"service-ca-operator-859f94c4b-bdnkb\" (UID: \"d763e40a-94ec-4167-8962-c355da8574d0\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bdnkb" Sep 10 13:21:10.050459 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.050009 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fe46372-65c2-451c-9fba-11399f8c753d-tmp-dir\") pod \"dns-default-2tqpb\" (UID: \"7fe46372-65c2-451c-9fba-11399f8c753d\") " pod="openshift-dns/dns-default-2tqpb" Sep 10 13:21:10.050459 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.050037 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4a051c0c-418c-439b-94e1-181c0a84e86b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-8k875\" (UID: \"4a051c0c-418c-439b-94e1-181c0a84e86b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8k875" Sep 10 13:21:10.050459 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.050054 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3ef3618e-e54b-46bf-8347-69b8e72720fd-registry-certificates\") pod \"image-registry-7d5587b5d-tzd9q\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" Sep 10 13:21:10.050459 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.050062 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqqbs\" (UniqueName: \"kubernetes.io/projected/ce4cfb8d-701a-414f-99ac-5f73e57c81ab-kube-api-access-bqqbs\") pod \"volume-data-source-validator-695c895bd6-482xw\" (UID: \"ce4cfb8d-701a-414f-99ac-5f73e57c81ab\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-482xw" Sep 10 13:21:10.050459 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.050087 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbcrn\" (UniqueName: \"kubernetes.io/projected/3ef3618e-e54b-46bf-8347-69b8e72720fd-kube-api-access-tbcrn\") pod \"image-registry-7d5587b5d-tzd9q\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" Sep 10 13:21:10.050459 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.050114 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3ef3618e-e54b-46bf-8347-69b8e72720fd-trusted-ca\") pod \"image-registry-7d5587b5d-tzd9q\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" Sep 10 13:21:10.050459 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.050140 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkrf7\" (UniqueName: \"kubernetes.io/projected/7fe46372-65c2-451c-9fba-11399f8c753d-kube-api-access-dkrf7\") pod \"dns-default-2tqpb\" (UID: \"7fe46372-65c2-451c-9fba-11399f8c753d\") " pod="openshift-dns/dns-default-2tqpb" Sep 10 13:21:10.050459 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.050170 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3ef3618e-e54b-46bf-8347-69b8e72720fd-installation-pull-secrets\") pod \"image-registry-7d5587b5d-tzd9q\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" Sep 10 13:21:10.050459 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.050193 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-service-ca-bundle\") pod \"router-default-5787c86768-n9xzb\" (UID: \"b19de8c5-85f7-4f5e-9011-8f38d0b80a69\") " pod="openshift-ingress/router-default-5787c86768-n9xzb" Sep 10 13:21:10.050459 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.050223 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/74a231ea-0f6d-4d1a-9e98-b54fa96a73bb-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-qkdhm\" (UID: \"74a231ea-0f6d-4d1a-9e98-b54fa96a73bb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qkdhm" Sep 10 13:21:10.050459 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.050252 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9c97ef99-3ac3-4334-aba7-f9de4a6cccf6-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-4hmcg\" (UID: \"9c97ef99-3ac3-4334-aba7-f9de4a6cccf6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4hmcg" Sep 10 13:21:10.050459 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:10.050330 2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 13:21:10.050459 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.050332 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fe9733a-2777-4507-af4b-29f330350747-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-mnksz\" (UID: \"4fe9733a-2777-4507-af4b-29f330350747\") " pod="openshift-insights/insights-operator-76bcd6ccf5-mnksz" Sep 10 13:21:10.050459 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:10.050375 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a051c0c-418c-439b-94e1-181c0a84e86b-cluster-monitoring-operator-tls podName:4a051c0c-418c-439b-94e1-181c0a84e86b nodeName:}" failed. No retries permitted until 2026-09-10 13:21:10.550359438 +0000 UTC m=+180.445076853 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4a051c0c-418c-439b-94e1-181c0a84e86b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-8k875" (UID: "4a051c0c-418c-439b-94e1-181c0a84e86b") : secret "cluster-monitoring-operator-tls" not found Sep 10 13:21:10.051197 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.050386 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/52fceedd-00f2-4943-ae47-dec51216d8e4-config\") pod \"console-operator-85b454874b-gtqlh\" (UID: \"52fceedd-00f2-4943-ae47-dec51216d8e4\") " pod="openshift-console-operator/console-operator-85b454874b-gtqlh" Sep 10 13:21:10.051197 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.050577 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fe46372-65c2-451c-9fba-11399f8c753d-tmp-dir\") pod \"dns-default-2tqpb\" (UID: \"7fe46372-65c2-451c-9fba-11399f8c753d\") " pod="openshift-dns/dns-default-2tqpb" Sep 10 13:21:10.051197 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:10.050667 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-service-ca-bundle podName:b19de8c5-85f7-4f5e-9011-8f38d0b80a69 nodeName:}" failed. No retries permitted until 2026-09-10 13:21:10.550648803 +0000 UTC m=+180.445366222 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-service-ca-bundle") pod "router-default-5787c86768-n9xzb" (UID: "b19de8c5-85f7-4f5e-9011-8f38d0b80a69") : configmap references non-existent config key: service-ca.crt Sep 10 13:21:10.051197 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.050812 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e7fc744a-ed33-44a7-94f1-19bbd01b9c50-nginx-conf\") pod \"networking-console-plugin-79977f9844-qw7g8\" (UID: \"e7fc744a-ed33-44a7-94f1-19bbd01b9c50\") " pod="openshift-network-console/networking-console-plugin-79977f9844-qw7g8" Sep 10 13:21:10.051197 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.050820 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fe9733a-2777-4507-af4b-29f330350747-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-mnksz\" (UID: \"4fe9733a-2777-4507-af4b-29f330350747\") " pod="openshift-insights/insights-operator-76bcd6ccf5-mnksz" Sep 10 13:21:10.051934 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:10.051232 2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 13:21:10.051934 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:10.051824 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74a231ea-0f6d-4d1a-9e98-b54fa96a73bb-samples-operator-tls podName:74a231ea-0f6d-4d1a-9e98-b54fa96a73bb nodeName:}" failed. No retries permitted until 2026-09-10 13:21:10.551264448 +0000 UTC m=+180.445981863 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/74a231ea-0f6d-4d1a-9e98-b54fa96a73bb-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-qkdhm" (UID: "74a231ea-0f6d-4d1a-9e98-b54fa96a73bb") : secret "samples-operator-tls" not found Sep 10 13:21:10.051934 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.051825 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3ef3618e-e54b-46bf-8347-69b8e72720fd-trusted-ca\") pod \"image-registry-7d5587b5d-tzd9q\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" Sep 10 13:21:10.051934 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.051926 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7fe46372-65c2-451c-9fba-11399f8c753d-config-volume\") pod \"dns-default-2tqpb\" (UID: \"7fe46372-65c2-451c-9fba-11399f8c753d\") " pod="openshift-dns/dns-default-2tqpb" Sep 10 13:21:10.052192 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.051962 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/227a2714-52ab-48f6-904b-9e997ed5f3a3-cert\") pod \"ingress-canary-zgs9h\" (UID: \"227a2714-52ab-48f6-904b-9e997ed5f3a3\") " pod="openshift-ingress-canary/ingress-canary-zgs9h" Sep 10 13:21:10.052192 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.052005 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4fe9733a-2777-4507-af4b-29f330350747-serving-cert\") pod \"insights-operator-76bcd6ccf5-mnksz\" (UID: \"4fe9733a-2777-4507-af4b-29f330350747\") " pod="openshift-insights/insights-operator-76bcd6ccf5-mnksz" Sep 10 13:21:10.052192 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.052030 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c58mw\" (UniqueName: \"kubernetes.io/projected/52fceedd-00f2-4943-ae47-dec51216d8e4-kube-api-access-c58mw\") pod \"console-operator-85b454874b-gtqlh\" (UID: \"52fceedd-00f2-4943-ae47-dec51216d8e4\") " pod="openshift-console-operator/console-operator-85b454874b-gtqlh" Sep 10 13:21:10.052192 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.052060 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3ef3618e-e54b-46bf-8347-69b8e72720fd-bound-sa-token\") pod \"image-registry-7d5587b5d-tzd9q\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" Sep 10 13:21:10.052192 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.052084 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-metrics-certs\") pod \"router-default-5787c86768-n9xzb\" (UID: \"b19de8c5-85f7-4f5e-9011-8f38d0b80a69\") " pod="openshift-ingress/router-default-5787c86768-n9xzb" Sep 10 13:21:10.052192 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.052116 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9c97ef99-3ac3-4334-aba7-f9de4a6cccf6-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-4hmcg\" (UID: \"9c97ef99-3ac3-4334-aba7-f9de4a6cccf6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4hmcg" Sep 10 13:21:10.052192 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.052136 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d763e40a-94ec-4167-8962-c355da8574d0-config\") pod \"service-ca-operator-859f94c4b-bdnkb\" (UID: \"d763e40a-94ec-4167-8962-c355da8574d0\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bdnkb" Sep 10 13:21:10.052192 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.052144 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3ef3618e-e54b-46bf-8347-69b8e72720fd-ca-trust-extracted\") pod \"image-registry-7d5587b5d-tzd9q\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" Sep 10 13:21:10.052192 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.052173 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlvjh\" (UniqueName: \"kubernetes.io/projected/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-kube-api-access-wlvjh\") pod \"router-default-5787c86768-n9xzb\" (UID: \"b19de8c5-85f7-4f5e-9011-8f38d0b80a69\") " pod="openshift-ingress/router-default-5787c86768-n9xzb" Sep 10 13:21:10.052706 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.052201 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzrsc\" (UniqueName: \"kubernetes.io/projected/d763e40a-94ec-4167-8962-c355da8574d0-kube-api-access-nzrsc\") pod \"service-ca-operator-859f94c4b-bdnkb\" (UID: \"d763e40a-94ec-4167-8962-c355da8574d0\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bdnkb" Sep 10 13:21:10.052706 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.052227 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/52fceedd-00f2-4943-ae47-dec51216d8e4-serving-cert\") pod \"console-operator-85b454874b-gtqlh\" (UID: \"52fceedd-00f2-4943-ae47-dec51216d8e4\") " pod="openshift-console-operator/console-operator-85b454874b-gtqlh" Sep 10 13:21:10.052706 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.052252 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hp6l\" (UniqueName: \"kubernetes.io/projected/04022e0a-02cd-4c63-a372-6b9f600c80cd-kube-api-access-9hp6l\") pod \"network-check-source-685456655f-9zr8q\" (UID: \"04022e0a-02cd-4c63-a372-6b9f600c80cd\") " pod="openshift-network-diagnostics/network-check-source-685456655f-9zr8q" Sep 10 13:21:10.052706 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.052284 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4fe9733a-2777-4507-af4b-29f330350747-snapshots\") pod \"insights-operator-76bcd6ccf5-mnksz\" (UID: \"4fe9733a-2777-4507-af4b-29f330350747\") " pod="openshift-insights/insights-operator-76bcd6ccf5-mnksz" Sep 10 13:21:10.052706 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.052310 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7fe46372-65c2-451c-9fba-11399f8c753d-metrics-tls\") pod \"dns-default-2tqpb\" (UID: \"7fe46372-65c2-451c-9fba-11399f8c753d\") " pod="openshift-dns/dns-default-2tqpb" Sep 10 13:21:10.052706 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.052341 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3ef3618e-e54b-46bf-8347-69b8e72720fd-registry-tls\") pod \"image-registry-7d5587b5d-tzd9q\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" Sep 10 13:21:10.052706 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.052373 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4fe9733a-2777-4507-af4b-29f330350747-tmp\") pod \"insights-operator-76bcd6ccf5-mnksz\" (UID: \"4fe9733a-2777-4507-af4b-29f330350747\") " pod="openshift-insights/insights-operator-76bcd6ccf5-mnksz" Sep 10 13:21:10.052706 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.052417 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/4a051c0c-418c-439b-94e1-181c0a84e86b-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-8k875\" (UID: \"4a051c0c-418c-439b-94e1-181c0a84e86b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8k875" Sep 10 13:21:10.052706 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.052483 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9c97ef99-3ac3-4334-aba7-f9de4a6cccf6-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-4hmcg\" (UID: \"9c97ef99-3ac3-4334-aba7-f9de4a6cccf6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4hmcg" Sep 10 13:21:10.053146 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.052859 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7fe46372-65c2-451c-9fba-11399f8c753d-config-volume\") pod \"dns-default-2tqpb\" (UID: \"7fe46372-65c2-451c-9fba-11399f8c753d\") " pod="openshift-dns/dns-default-2tqpb" Sep 10 13:21:10.053146 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:10.053054 2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 10 13:21:10.053146 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:10.053113 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-metrics-certs podName:b19de8c5-85f7-4f5e-9011-8f38d0b80a69 nodeName:}" failed. No retries permitted until 2026-09-10 13:21:10.553084681 +0000 UTC m=+180.447802080 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-metrics-certs") pod "router-default-5787c86768-n9xzb" (UID: "b19de8c5-85f7-4f5e-9011-8f38d0b80a69") : secret "router-metrics-certs-default" not found Sep 10 13:21:10.053146 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.053129 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/52fceedd-00f2-4943-ae47-dec51216d8e4-trusted-ca\") pod \"console-operator-85b454874b-gtqlh\" (UID: \"52fceedd-00f2-4943-ae47-dec51216d8e4\") " pod="openshift-console-operator/console-operator-85b454874b-gtqlh" Sep 10 13:21:10.053360 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.053136 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/4a051c0c-418c-439b-94e1-181c0a84e86b-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-8k875\" (UID: \"4a051c0c-418c-439b-94e1-181c0a84e86b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8k875" Sep 10 13:21:10.053360 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:10.053236 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 13:21:10.053360 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:10.053289 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fe46372-65c2-451c-9fba-11399f8c753d-metrics-tls podName:7fe46372-65c2-451c-9fba-11399f8c753d nodeName:}" failed. No retries permitted until 2026-09-10 13:21:10.553274527 +0000 UTC m=+180.447991929 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7fe46372-65c2-451c-9fba-11399f8c753d-metrics-tls") pod "dns-default-2tqpb" (UID: "7fe46372-65c2-451c-9fba-11399f8c753d") : secret "dns-default-metrics-tls" not found Sep 10 13:21:10.053630 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:10.053612 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 13:21:10.053707 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:10.053632 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d5587b5d-tzd9q: secret "image-registry-tls" not found Sep 10 13:21:10.053707 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:10.053672 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3ef3618e-e54b-46bf-8347-69b8e72720fd-registry-tls podName:3ef3618e-e54b-46bf-8347-69b8e72720fd nodeName:}" failed. No retries permitted until 2026-09-10 13:21:10.553658943 +0000 UTC m=+180.448376346 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3ef3618e-e54b-46bf-8347-69b8e72720fd-registry-tls") pod "image-registry-7d5587b5d-tzd9q" (UID: "3ef3618e-e54b-46bf-8347-69b8e72720fd") : secret "image-registry-tls" not found Sep 10 13:21:10.053819 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.053783 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4fe9733a-2777-4507-af4b-29f330350747-snapshots\") pod \"insights-operator-76bcd6ccf5-mnksz\" (UID: \"4fe9733a-2777-4507-af4b-29f330350747\") " pod="openshift-insights/insights-operator-76bcd6ccf5-mnksz" Sep 10 13:21:10.054073 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.054014 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3ef3618e-e54b-46bf-8347-69b8e72720fd-ca-trust-extracted\") pod \"image-registry-7d5587b5d-tzd9q\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" Sep 10 13:21:10.054417 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.054230 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4fe9733a-2777-4507-af4b-29f330350747-tmp\") pod \"insights-operator-76bcd6ccf5-mnksz\" (UID: \"4fe9733a-2777-4507-af4b-29f330350747\") " pod="openshift-insights/insights-operator-76bcd6ccf5-mnksz" Sep 10 13:21:10.054576 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.054544 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d763e40a-94ec-4167-8962-c355da8574d0-serving-cert\") pod \"service-ca-operator-859f94c4b-bdnkb\" (UID: \"d763e40a-94ec-4167-8962-c355da8574d0\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bdnkb" Sep 10 13:21:10.054780 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.054750 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-stats-auth\") pod \"router-default-5787c86768-n9xzb\" (UID: \"b19de8c5-85f7-4f5e-9011-8f38d0b80a69\") " pod="openshift-ingress/router-default-5787c86768-n9xzb" Sep 10 13:21:10.055631 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.055597 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/227a2714-52ab-48f6-904b-9e997ed5f3a3-cert\") pod \"ingress-canary-zgs9h\" (UID: \"227a2714-52ab-48f6-904b-9e997ed5f3a3\") " pod="openshift-ingress-canary/ingress-canary-zgs9h" Sep 10 13:21:10.055857 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.055833 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3ef3618e-e54b-46bf-8347-69b8e72720fd-image-registry-private-configuration\") pod \"image-registry-7d5587b5d-tzd9q\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" Sep 10 13:21:10.056496 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.056470 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9c97ef99-3ac3-4334-aba7-f9de4a6cccf6-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-4hmcg\" (UID: \"9c97ef99-3ac3-4334-aba7-f9de4a6cccf6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4hmcg" Sep 10 13:21:10.056608 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.056547 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3ef3618e-e54b-46bf-8347-69b8e72720fd-installation-pull-secrets\") pod \"image-registry-7d5587b5d-tzd9q\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" Sep 10 13:21:10.056819 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.056800 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/52fceedd-00f2-4943-ae47-dec51216d8e4-serving-cert\") pod \"console-operator-85b454874b-gtqlh\" (UID: \"52fceedd-00f2-4943-ae47-dec51216d8e4\") " pod="openshift-console-operator/console-operator-85b454874b-gtqlh" Sep 10 13:21:10.056878 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.056820 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-default-certificate\") pod \"router-default-5787c86768-n9xzb\" (UID: \"b19de8c5-85f7-4f5e-9011-8f38d0b80a69\") " pod="openshift-ingress/router-default-5787c86768-n9xzb" Sep 10 13:21:10.057100 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.057080 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4fe9733a-2777-4507-af4b-29f330350747-serving-cert\") pod \"insights-operator-76bcd6ccf5-mnksz\" (UID: \"4fe9733a-2777-4507-af4b-29f330350747\") " pod="openshift-insights/insights-operator-76bcd6ccf5-mnksz" Sep 10 13:21:10.061598 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.061576 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnl9t\" (UniqueName: \"kubernetes.io/projected/4a051c0c-418c-439b-94e1-181c0a84e86b-kube-api-access-xnl9t\") pod \"cluster-monitoring-operator-6cdcc96965-8k875\" (UID: \"4a051c0c-418c-439b-94e1-181c0a84e86b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8k875" Sep 10 13:21:10.062616 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.062590 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkrf7\" (UniqueName: \"kubernetes.io/projected/7fe46372-65c2-451c-9fba-11399f8c753d-kube-api-access-dkrf7\") pod \"dns-default-2tqpb\" (UID: \"7fe46372-65c2-451c-9fba-11399f8c753d\") " pod="openshift-dns/dns-default-2tqpb" Sep 10 13:21:10.062824 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.062799 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5djq\" (UniqueName: \"kubernetes.io/projected/227a2714-52ab-48f6-904b-9e997ed5f3a3-kube-api-access-l5djq\") pod \"ingress-canary-zgs9h\" (UID: \"227a2714-52ab-48f6-904b-9e997ed5f3a3\") " pod="openshift-ingress-canary/ingress-canary-zgs9h" Sep 10 13:21:10.063550 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.063525 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqqbs\" (UniqueName: \"kubernetes.io/projected/ce4cfb8d-701a-414f-99ac-5f73e57c81ab-kube-api-access-bqqbs\") pod \"volume-data-source-validator-695c895bd6-482xw\" (UID: \"ce4cfb8d-701a-414f-99ac-5f73e57c81ab\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-482xw" Sep 10 13:21:10.064137 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.064115 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbcrn\" (UniqueName: \"kubernetes.io/projected/3ef3618e-e54b-46bf-8347-69b8e72720fd-kube-api-access-tbcrn\") pod \"image-registry-7d5587b5d-tzd9q\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" Sep 10 13:21:10.065310 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.065289 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfld6\" (UniqueName: \"kubernetes.io/projected/9c97ef99-3ac3-4334-aba7-f9de4a6cccf6-kube-api-access-nfld6\") pod \"kube-storage-version-migrator-operator-8d7dd4675-4hmcg\" (UID: \"9c97ef99-3ac3-4334-aba7-f9de4a6cccf6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4hmcg" Sep 10 13:21:10.065462 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.065446 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdfg8\" (UniqueName: \"kubernetes.io/projected/4fe9733a-2777-4507-af4b-29f330350747-kube-api-access-tdfg8\") pod \"insights-operator-76bcd6ccf5-mnksz\" (UID: \"4fe9733a-2777-4507-af4b-29f330350747\") " pod="openshift-insights/insights-operator-76bcd6ccf5-mnksz" Sep 10 13:21:10.068464 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.068441 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjxrf\" (UniqueName: \"kubernetes.io/projected/74a231ea-0f6d-4d1a-9e98-b54fa96a73bb-kube-api-access-cjxrf\") pod \"cluster-samples-operator-7cdcfcf76c-qkdhm\" (UID: \"74a231ea-0f6d-4d1a-9e98-b54fa96a73bb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qkdhm" Sep 10 13:21:10.068939 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.068916 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c58mw\" (UniqueName: \"kubernetes.io/projected/52fceedd-00f2-4943-ae47-dec51216d8e4-kube-api-access-c58mw\") pod \"console-operator-85b454874b-gtqlh\" (UID: \"52fceedd-00f2-4943-ae47-dec51216d8e4\") " pod="openshift-console-operator/console-operator-85b454874b-gtqlh" Sep 10 13:21:10.069226 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.069204 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hp6l\" (UniqueName: \"kubernetes.io/projected/04022e0a-02cd-4c63-a372-6b9f600c80cd-kube-api-access-9hp6l\") pod \"network-check-source-685456655f-9zr8q\" (UID: \"04022e0a-02cd-4c63-a372-6b9f600c80cd\") " pod="openshift-network-diagnostics/network-check-source-685456655f-9zr8q" Sep 10 13:21:10.070800 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.070768 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlvjh\" (UniqueName: \"kubernetes.io/projected/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-kube-api-access-wlvjh\") pod \"router-default-5787c86768-n9xzb\" (UID: \"b19de8c5-85f7-4f5e-9011-8f38d0b80a69\") " pod="openshift-ingress/router-default-5787c86768-n9xzb" Sep 10 13:21:10.071762 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.071741 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3ef3618e-e54b-46bf-8347-69b8e72720fd-bound-sa-token\") pod \"image-registry-7d5587b5d-tzd9q\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" Sep 10 13:21:10.071900 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.071884 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzrsc\" (UniqueName: \"kubernetes.io/projected/d763e40a-94ec-4167-8962-c355da8574d0-kube-api-access-nzrsc\") pod \"service-ca-operator-859f94c4b-bdnkb\" (UID: \"d763e40a-94ec-4167-8962-c355da8574d0\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bdnkb" Sep 10 13:21:10.102487 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.102455 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-mnksz" Sep 10 13:21:10.109386 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.109307 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-482xw" Sep 10 13:21:10.130416 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.130370 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-gtqlh" Sep 10 13:21:10.137272 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.137244 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4hmcg" Sep 10 13:21:10.149528 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.149500 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bdnkb" Sep 10 13:21:10.158750 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.158701 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-9zr8q" Sep 10 13:21:10.181443 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.180964 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zgs9h" Sep 10 13:21:10.403055 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.400948 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-482xw"] Sep 10 13:21:10.405996 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:21:10.405970 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce4cfb8d_701a_414f_99ac_5f73e57c81ab.slice/crio-70e2a9237e42ce1f5ce3cc0f524f754a2bf34ab2a78a3403944a24fa44981034 WatchSource:0}: Error finding container 70e2a9237e42ce1f5ce3cc0f524f754a2bf34ab2a78a3403944a24fa44981034: Status 404 returned error can't find the container with id 70e2a9237e42ce1f5ce3cc0f524f754a2bf34ab2a78a3403944a24fa44981034 Sep 10 13:21:10.427217 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.427193 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-mnksz"] Sep 10 13:21:10.429763 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:21:10.429739 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fe9733a_2777_4507_af4b_29f330350747.slice/crio-0017f24b2c56c228639a0c801b26946345b2c738c9a84ee856d15b84545b4860 WatchSource:0}: Error finding container 0017f24b2c56c228639a0c801b26946345b2c738c9a84ee856d15b84545b4860: Status 404 returned error can't find the container with id 0017f24b2c56c228639a0c801b26946345b2c738c9a84ee856d15b84545b4860 Sep 10 13:21:10.461572 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.461543 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-gtqlh"] Sep 10 13:21:10.471166 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:21:10.471138 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52fceedd_00f2_4943_ae47_dec51216d8e4.slice/crio-2d45f5c95ef9392ec471a03deeda9c9d2417827d43fe268978e83485b112285a WatchSource:0}: Error finding container 2d45f5c95ef9392ec471a03deeda9c9d2417827d43fe268978e83485b112285a: Status 404 returned error can't find the container with id 2d45f5c95ef9392ec471a03deeda9c9d2417827d43fe268978e83485b112285a Sep 10 13:21:10.557325 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.557290 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4a051c0c-418c-439b-94e1-181c0a84e86b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-8k875\" (UID: \"4a051c0c-418c-439b-94e1-181c0a84e86b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8k875" Sep 10 13:21:10.557507 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.557344 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-service-ca-bundle\") pod \"router-default-5787c86768-n9xzb\" (UID: \"b19de8c5-85f7-4f5e-9011-8f38d0b80a69\") " pod="openshift-ingress/router-default-5787c86768-n9xzb" Sep 10 13:21:10.557507 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:10.557474 2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 13:21:10.557507 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:10.557491 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-service-ca-bundle podName:b19de8c5-85f7-4f5e-9011-8f38d0b80a69 nodeName:}" failed. No retries permitted until 2026-09-10 13:21:11.557472768 +0000 UTC m=+181.452190179 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-service-ca-bundle") pod "router-default-5787c86768-n9xzb" (UID: "b19de8c5-85f7-4f5e-9011-8f38d0b80a69") : configmap references non-existent config key: service-ca.crt Sep 10 13:21:10.557679 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.557473 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/74a231ea-0f6d-4d1a-9e98-b54fa96a73bb-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-qkdhm\" (UID: \"74a231ea-0f6d-4d1a-9e98-b54fa96a73bb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qkdhm" Sep 10 13:21:10.557679 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:10.557537 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a051c0c-418c-439b-94e1-181c0a84e86b-cluster-monitoring-operator-tls podName:4a051c0c-418c-439b-94e1-181c0a84e86b nodeName:}" failed. No retries permitted until 2026-09-10 13:21:11.557519542 +0000 UTC m=+181.452236938 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4a051c0c-418c-439b-94e1-181c0a84e86b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-8k875" (UID: "4a051c0c-418c-439b-94e1-181c0a84e86b") : secret "cluster-monitoring-operator-tls" not found Sep 10 13:21:10.557679 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:10.557535 2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 13:21:10.557679 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:10.557606 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74a231ea-0f6d-4d1a-9e98-b54fa96a73bb-samples-operator-tls podName:74a231ea-0f6d-4d1a-9e98-b54fa96a73bb nodeName:}" failed. No retries permitted until 2026-09-10 13:21:11.557586104 +0000 UTC m=+181.452303519 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/74a231ea-0f6d-4d1a-9e98-b54fa96a73bb-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-qkdhm" (UID: "74a231ea-0f6d-4d1a-9e98-b54fa96a73bb") : secret "samples-operator-tls" not found Sep 10 13:21:10.557679 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.557642 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-metrics-certs\") pod \"router-default-5787c86768-n9xzb\" (UID: \"b19de8c5-85f7-4f5e-9011-8f38d0b80a69\") " pod="openshift-ingress/router-default-5787c86768-n9xzb" Sep 10 13:21:10.557939 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.557691 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7fe46372-65c2-451c-9fba-11399f8c753d-metrics-tls\") pod \"dns-default-2tqpb\" (UID: \"7fe46372-65c2-451c-9fba-11399f8c753d\") " pod="openshift-dns/dns-default-2tqpb" Sep 10 13:21:10.557939 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.557721 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3ef3618e-e54b-46bf-8347-69b8e72720fd-registry-tls\") pod \"image-registry-7d5587b5d-tzd9q\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" Sep 10 13:21:10.557939 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:10.557766 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 13:21:10.557939 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:10.557781 2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 10 13:21:10.557939 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:10.557815 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fe46372-65c2-451c-9fba-11399f8c753d-metrics-tls podName:7fe46372-65c2-451c-9fba-11399f8c753d nodeName:}" failed. No retries permitted until 2026-09-10 13:21:11.557804216 +0000 UTC m=+181.452521625 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7fe46372-65c2-451c-9fba-11399f8c753d-metrics-tls") pod "dns-default-2tqpb" (UID: "7fe46372-65c2-451c-9fba-11399f8c753d") : secret "dns-default-metrics-tls" not found Sep 10 13:21:10.557939 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:10.557816 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 13:21:10.557939 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:10.557829 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-metrics-certs podName:b19de8c5-85f7-4f5e-9011-8f38d0b80a69 nodeName:}" failed. No retries permitted until 2026-09-10 13:21:11.5578234 +0000 UTC m=+181.452540795 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-metrics-certs") pod "router-default-5787c86768-n9xzb" (UID: "b19de8c5-85f7-4f5e-9011-8f38d0b80a69") : secret "router-metrics-certs-default" not found Sep 10 13:21:10.557939 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:10.557830 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d5587b5d-tzd9q: secret "image-registry-tls" not found Sep 10 13:21:10.557939 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:10.557870 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3ef3618e-e54b-46bf-8347-69b8e72720fd-registry-tls podName:3ef3618e-e54b-46bf-8347-69b8e72720fd nodeName:}" failed. No retries permitted until 2026-09-10 13:21:11.557862259 +0000 UTC m=+181.452579659 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3ef3618e-e54b-46bf-8347-69b8e72720fd-registry-tls") pod "image-registry-7d5587b5d-tzd9q" (UID: "3ef3618e-e54b-46bf-8347-69b8e72720fd") : secret "image-registry-tls" not found Sep 10 13:21:10.557939 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.557898 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e7fc744a-ed33-44a7-94f1-19bbd01b9c50-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-qw7g8\" (UID: \"e7fc744a-ed33-44a7-94f1-19bbd01b9c50\") " pod="openshift-network-console/networking-console-plugin-79977f9844-qw7g8" Sep 10 13:21:10.558224 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:10.557976 2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 13:21:10.558224 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:10.558026 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7fc744a-ed33-44a7-94f1-19bbd01b9c50-networking-console-plugin-cert podName:e7fc744a-ed33-44a7-94f1-19bbd01b9c50 nodeName:}" failed. No retries permitted until 2026-09-10 13:21:11.558013923 +0000 UTC m=+181.452731326 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e7fc744a-ed33-44a7-94f1-19bbd01b9c50-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-qw7g8" (UID: "e7fc744a-ed33-44a7-94f1-19bbd01b9c50") : secret "networking-console-plugin-cert" not found Sep 10 13:21:10.563635 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.563613 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zgs9h"] Sep 10 13:21:10.568158 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:21:10.568134 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod227a2714_52ab_48f6_904b_9e997ed5f3a3.slice/crio-48a90c5b4dd8913e21d71edc153c8ce036afa7b691871206737c57de1b8d8c51 WatchSource:0}: Error finding container 48a90c5b4dd8913e21d71edc153c8ce036afa7b691871206737c57de1b8d8c51: Status 404 returned error can't find the container with id 48a90c5b4dd8913e21d71edc153c8ce036afa7b691871206737c57de1b8d8c51 Sep 10 13:21:10.599320 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.599285 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4hmcg"] Sep 10 13:21:10.605875 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:21:10.605852 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c97ef99_3ac3_4334_aba7_f9de4a6cccf6.slice/crio-50d7fb96038157fe0c7fd021f255911f8df8516dec103c9b5ae0f62dbceaf5ff WatchSource:0}: Error finding container 50d7fb96038157fe0c7fd021f255911f8df8516dec103c9b5ae0f62dbceaf5ff: Status 404 returned error can't find the container with id 50d7fb96038157fe0c7fd021f255911f8df8516dec103c9b5ae0f62dbceaf5ff Sep 10 13:21:10.607844 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.607786 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-9zr8q"] Sep 10 13:21:10.609155 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:21:10.609136 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04022e0a_02cd_4c63_a372_6b9f600c80cd.slice/crio-d2617ed0b2087861d2b3e570cbf0a2c3930ddcc11057a28362a20f91f12d447f WatchSource:0}: Error finding container d2617ed0b2087861d2b3e570cbf0a2c3930ddcc11057a28362a20f91f12d447f: Status 404 returned error can't find the container with id d2617ed0b2087861d2b3e570cbf0a2c3930ddcc11057a28362a20f91f12d447f Sep 10 13:21:10.642421 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.642379 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-bdnkb"] Sep 10 13:21:10.669330 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:21:10.669303 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd763e40a_94ec_4167_8962_c355da8574d0.slice/crio-11ffee09b205ac3887f714e8f838e6b38a41c57e0578e5aa30a09f2e20e57f95 WatchSource:0}: Error finding container 11ffee09b205ac3887f714e8f838e6b38a41c57e0578e5aa30a09f2e20e57f95: Status 404 returned error can't find the container with id 11ffee09b205ac3887f714e8f838e6b38a41c57e0578e5aa30a09f2e20e57f95 Sep 10 13:21:10.958817 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.958735 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zgs9h" event={"ID":"227a2714-52ab-48f6-904b-9e997ed5f3a3","Type":"ContainerStarted","Data":"48a90c5b4dd8913e21d71edc153c8ce036afa7b691871206737c57de1b8d8c51"} Sep 10 13:21:10.959866 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.959833 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-482xw" event={"ID":"ce4cfb8d-701a-414f-99ac-5f73e57c81ab","Type":"ContainerStarted","Data":"70e2a9237e42ce1f5ce3cc0f524f754a2bf34ab2a78a3403944a24fa44981034"} Sep 10 13:21:10.960766 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.960743 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bdnkb" event={"ID":"d763e40a-94ec-4167-8962-c355da8574d0","Type":"ContainerStarted","Data":"11ffee09b205ac3887f714e8f838e6b38a41c57e0578e5aa30a09f2e20e57f95"} Sep 10 13:21:10.961846 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.961818 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-mnksz" event={"ID":"4fe9733a-2777-4507-af4b-29f330350747","Type":"ContainerStarted","Data":"0017f24b2c56c228639a0c801b26946345b2c738c9a84ee856d15b84545b4860"} Sep 10 13:21:10.962815 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.962796 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-9zr8q" event={"ID":"04022e0a-02cd-4c63-a372-6b9f600c80cd","Type":"ContainerStarted","Data":"d2617ed0b2087861d2b3e570cbf0a2c3930ddcc11057a28362a20f91f12d447f"} Sep 10 13:21:10.963790 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.963770 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-gtqlh" event={"ID":"52fceedd-00f2-4943-ae47-dec51216d8e4","Type":"ContainerStarted","Data":"2d45f5c95ef9392ec471a03deeda9c9d2417827d43fe268978e83485b112285a"} Sep 10 13:21:10.964814 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:10.964796 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4hmcg" event={"ID":"9c97ef99-3ac3-4334-aba7-f9de4a6cccf6","Type":"ContainerStarted","Data":"50d7fb96038157fe0c7fd021f255911f8df8516dec103c9b5ae0f62dbceaf5ff"} Sep 10 13:21:11.566157 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:11.566120 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4a051c0c-418c-439b-94e1-181c0a84e86b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-8k875\" (UID: \"4a051c0c-418c-439b-94e1-181c0a84e86b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8k875" Sep 10 13:21:11.566301 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:11.566179 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-service-ca-bundle\") pod \"router-default-5787c86768-n9xzb\" (UID: \"b19de8c5-85f7-4f5e-9011-8f38d0b80a69\") " pod="openshift-ingress/router-default-5787c86768-n9xzb" Sep 10 13:21:11.566301 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:11.566208 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/74a231ea-0f6d-4d1a-9e98-b54fa96a73bb-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-qkdhm\" (UID: \"74a231ea-0f6d-4d1a-9e98-b54fa96a73bb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qkdhm" Sep 10 13:21:11.566301 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:11.566260 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-metrics-certs\") pod \"router-default-5787c86768-n9xzb\" (UID: \"b19de8c5-85f7-4f5e-9011-8f38d0b80a69\") " pod="openshift-ingress/router-default-5787c86768-n9xzb" Sep 10 13:21:11.566301 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:11.566297 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7fe46372-65c2-451c-9fba-11399f8c753d-metrics-tls\") pod \"dns-default-2tqpb\" (UID: \"7fe46372-65c2-451c-9fba-11399f8c753d\") " pod="openshift-dns/dns-default-2tqpb" Sep 10 13:21:11.566536 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:11.566322 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3ef3618e-e54b-46bf-8347-69b8e72720fd-registry-tls\") pod \"image-registry-7d5587b5d-tzd9q\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" Sep 10 13:21:11.566536 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:11.566384 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e7fc744a-ed33-44a7-94f1-19bbd01b9c50-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-qw7g8\" (UID: \"e7fc744a-ed33-44a7-94f1-19bbd01b9c50\") " pod="openshift-network-console/networking-console-plugin-79977f9844-qw7g8" Sep 10 13:21:11.566536 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:11.566529 2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 13:21:11.566689 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:11.566590 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7fc744a-ed33-44a7-94f1-19bbd01b9c50-networking-console-plugin-cert podName:e7fc744a-ed33-44a7-94f1-19bbd01b9c50 nodeName:}" failed. No retries permitted until 2026-09-10 13:21:13.566571467 +0000 UTC m=+183.461288876 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e7fc744a-ed33-44a7-94f1-19bbd01b9c50-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-qw7g8" (UID: "e7fc744a-ed33-44a7-94f1-19bbd01b9c50") : secret "networking-console-plugin-cert" not found Sep 10 13:21:11.567497 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:11.566978 2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 13:21:11.567497 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:11.567025 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a051c0c-418c-439b-94e1-181c0a84e86b-cluster-monitoring-operator-tls podName:4a051c0c-418c-439b-94e1-181c0a84e86b nodeName:}" failed. No retries permitted until 2026-09-10 13:21:13.567011946 +0000 UTC m=+183.461729356 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4a051c0c-418c-439b-94e1-181c0a84e86b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-8k875" (UID: "4a051c0c-418c-439b-94e1-181c0a84e86b") : secret "cluster-monitoring-operator-tls" not found Sep 10 13:21:11.567497 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:11.567096 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-service-ca-bundle podName:b19de8c5-85f7-4f5e-9011-8f38d0b80a69 nodeName:}" failed. No retries permitted until 2026-09-10 13:21:13.567086678 +0000 UTC m=+183.461804089 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-service-ca-bundle") pod "router-default-5787c86768-n9xzb" (UID: "b19de8c5-85f7-4f5e-9011-8f38d0b80a69") : configmap references non-existent config key: service-ca.crt Sep 10 13:21:11.567497 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:11.567146 2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 13:21:11.567497 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:11.567174 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74a231ea-0f6d-4d1a-9e98-b54fa96a73bb-samples-operator-tls podName:74a231ea-0f6d-4d1a-9e98-b54fa96a73bb nodeName:}" failed. No retries permitted until 2026-09-10 13:21:13.567165004 +0000 UTC m=+183.461882403 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/74a231ea-0f6d-4d1a-9e98-b54fa96a73bb-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-qkdhm" (UID: "74a231ea-0f6d-4d1a-9e98-b54fa96a73bb") : secret "samples-operator-tls" not found Sep 10 13:21:11.567497 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:11.567238 2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 10 13:21:11.567497 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:11.567265 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-metrics-certs podName:b19de8c5-85f7-4f5e-9011-8f38d0b80a69 nodeName:}" failed. No retries permitted until 2026-09-10 13:21:13.56725605 +0000 UTC m=+183.461973450 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-metrics-certs") pod "router-default-5787c86768-n9xzb" (UID: "b19de8c5-85f7-4f5e-9011-8f38d0b80a69") : secret "router-metrics-certs-default" not found Sep 10 13:21:11.567497 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:11.567315 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 13:21:11.567497 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:11.567341 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fe46372-65c2-451c-9fba-11399f8c753d-metrics-tls podName:7fe46372-65c2-451c-9fba-11399f8c753d nodeName:}" failed. No retries permitted until 2026-09-10 13:21:13.567332095 +0000 UTC m=+183.462049495 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7fe46372-65c2-451c-9fba-11399f8c753d-metrics-tls") pod "dns-default-2tqpb" (UID: "7fe46372-65c2-451c-9fba-11399f8c753d") : secret "dns-default-metrics-tls" not found Sep 10 13:21:11.567497 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:11.567397 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 13:21:11.567497 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:11.567424 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d5587b5d-tzd9q: secret "image-registry-tls" not found Sep 10 13:21:11.567497 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:11.567453 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3ef3618e-e54b-46bf-8347-69b8e72720fd-registry-tls podName:3ef3618e-e54b-46bf-8347-69b8e72720fd nodeName:}" failed. No retries permitted until 2026-09-10 13:21:13.567443531 +0000 UTC m=+183.462160933 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3ef3618e-e54b-46bf-8347-69b8e72720fd-registry-tls") pod "image-registry-7d5587b5d-tzd9q" (UID: "3ef3618e-e54b-46bf-8347-69b8e72720fd") : secret "image-registry-tls" not found Sep 10 13:21:13.586311 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:13.586259 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e7fc744a-ed33-44a7-94f1-19bbd01b9c50-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-qw7g8\" (UID: \"e7fc744a-ed33-44a7-94f1-19bbd01b9c50\") " pod="openshift-network-console/networking-console-plugin-79977f9844-qw7g8" Sep 10 13:21:13.586763 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:13.586336 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4a051c0c-418c-439b-94e1-181c0a84e86b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-8k875\" (UID: \"4a051c0c-418c-439b-94e1-181c0a84e86b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8k875" Sep 10 13:21:13.586763 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:13.586366 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-service-ca-bundle\") pod \"router-default-5787c86768-n9xzb\" (UID: \"b19de8c5-85f7-4f5e-9011-8f38d0b80a69\") " pod="openshift-ingress/router-default-5787c86768-n9xzb" Sep 10 13:21:13.586763 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:13.586390 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/74a231ea-0f6d-4d1a-9e98-b54fa96a73bb-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-qkdhm\" (UID: \"74a231ea-0f6d-4d1a-9e98-b54fa96a73bb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qkdhm" Sep 10 13:21:13.586763 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:13.586443 2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 13:21:13.586763 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:13.586464 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-metrics-certs\") pod \"router-default-5787c86768-n9xzb\" (UID: \"b19de8c5-85f7-4f5e-9011-8f38d0b80a69\") " pod="openshift-ingress/router-default-5787c86768-n9xzb" Sep 10 13:21:13.586763 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:13.586506 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7fe46372-65c2-451c-9fba-11399f8c753d-metrics-tls\") pod \"dns-default-2tqpb\" (UID: \"7fe46372-65c2-451c-9fba-11399f8c753d\") " pod="openshift-dns/dns-default-2tqpb" Sep 10 13:21:13.586763 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:13.586514 2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 13:21:13.586763 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:13.586523 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7fc744a-ed33-44a7-94f1-19bbd01b9c50-networking-console-plugin-cert podName:e7fc744a-ed33-44a7-94f1-19bbd01b9c50 nodeName:}" failed. No retries permitted until 2026-09-10 13:21:17.586500926 +0000 UTC m=+187.481218321 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e7fc744a-ed33-44a7-94f1-19bbd01b9c50-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-qw7g8" (UID: "e7fc744a-ed33-44a7-94f1-19bbd01b9c50") : secret "networking-console-plugin-cert" not found Sep 10 13:21:13.586763 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:13.586581 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 13:21:13.586763 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:13.586593 2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 13:21:13.586763 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:13.586597 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-service-ca-bundle podName:b19de8c5-85f7-4f5e-9011-8f38d0b80a69 nodeName:}" failed. No retries permitted until 2026-09-10 13:21:17.586575785 +0000 UTC m=+187.481293184 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-service-ca-bundle") pod "router-default-5787c86768-n9xzb" (UID: "b19de8c5-85f7-4f5e-9011-8f38d0b80a69") : configmap references non-existent config key: service-ca.crt Sep 10 13:21:13.586763 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:13.586621 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a051c0c-418c-439b-94e1-181c0a84e86b-cluster-monitoring-operator-tls podName:4a051c0c-418c-439b-94e1-181c0a84e86b nodeName:}" failed. No retries permitted until 2026-09-10 13:21:17.586610705 +0000 UTC m=+187.481328103 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4a051c0c-418c-439b-94e1-181c0a84e86b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-8k875" (UID: "4a051c0c-418c-439b-94e1-181c0a84e86b") : secret "cluster-monitoring-operator-tls" not found Sep 10 13:21:13.586763 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:13.586627 2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 10 13:21:13.586763 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:13.586649 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fe46372-65c2-451c-9fba-11399f8c753d-metrics-tls podName:7fe46372-65c2-451c-9fba-11399f8c753d nodeName:}" failed. No retries permitted until 2026-09-10 13:21:17.586638706 +0000 UTC m=+187.481356106 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7fe46372-65c2-451c-9fba-11399f8c753d-metrics-tls") pod "dns-default-2tqpb" (UID: "7fe46372-65c2-451c-9fba-11399f8c753d") : secret "dns-default-metrics-tls" not found Sep 10 13:21:13.586763 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:13.586662 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74a231ea-0f6d-4d1a-9e98-b54fa96a73bb-samples-operator-tls podName:74a231ea-0f6d-4d1a-9e98-b54fa96a73bb nodeName:}" failed. No retries permitted until 2026-09-10 13:21:17.586656358 +0000 UTC m=+187.481373756 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/74a231ea-0f6d-4d1a-9e98-b54fa96a73bb-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-qkdhm" (UID: "74a231ea-0f6d-4d1a-9e98-b54fa96a73bb") : secret "samples-operator-tls" not found Sep 10 13:21:13.587342 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:13.586674 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3ef3618e-e54b-46bf-8347-69b8e72720fd-registry-tls\") pod \"image-registry-7d5587b5d-tzd9q\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" Sep 10 13:21:13.587342 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:13.586714 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-metrics-certs podName:b19de8c5-85f7-4f5e-9011-8f38d0b80a69 nodeName:}" failed. No retries permitted until 2026-09-10 13:21:17.586708689 +0000 UTC m=+187.481426091 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-metrics-certs") pod "router-default-5787c86768-n9xzb" (UID: "b19de8c5-85f7-4f5e-9011-8f38d0b80a69") : secret "router-metrics-certs-default" not found Sep 10 13:21:13.587342 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:13.586721 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 13:21:13.587342 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:13.586732 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d5587b5d-tzd9q: secret "image-registry-tls" not found Sep 10 13:21:13.587342 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:13.586774 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3ef3618e-e54b-46bf-8347-69b8e72720fd-registry-tls podName:3ef3618e-e54b-46bf-8347-69b8e72720fd nodeName:}" failed. No retries permitted until 2026-09-10 13:21:17.586752636 +0000 UTC m=+187.481470037 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3ef3618e-e54b-46bf-8347-69b8e72720fd-registry-tls") pod "image-registry-7d5587b5d-tzd9q" (UID: "3ef3618e-e54b-46bf-8347-69b8e72720fd") : secret "image-registry-tls" not found Sep 10 13:21:15.083125 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:15.083087 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:15.209324 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:15.209288 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:16.987091 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:16.987048 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zgs9h" event={"ID":"227a2714-52ab-48f6-904b-9e997ed5f3a3","Type":"ContainerStarted","Data":"2d971a4b5496df57c9456a1d743239053be1db3e5720edcb8f61c6ba5cfc1ef1"} Sep 10 13:21:16.988922 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:16.988836 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-482xw" event={"ID":"ce4cfb8d-701a-414f-99ac-5f73e57c81ab","Type":"ContainerStarted","Data":"f2cb7cf534a76a31fa89310f0c491e970d99b395a38bd22cecf21ab66910a5c8"} Sep 10 13:21:16.990547 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:16.990522 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bdnkb" event={"ID":"d763e40a-94ec-4167-8962-c355da8574d0","Type":"ContainerStarted","Data":"b1c768f0e64cbdcbc26be0e8add260c967b2a033527dc195cc81ade87e2139ee"} Sep 10 13:21:16.992139 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:16.992118 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-mnksz" event={"ID":"4fe9733a-2777-4507-af4b-29f330350747","Type":"ContainerStarted","Data":"50865a8e04de1851b987b68f47802430a3bb87d07940542a56ac04af290a751c"} Sep 10 13:21:16.994023 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:16.993717 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-9zr8q" event={"ID":"04022e0a-02cd-4c63-a372-6b9f600c80cd","Type":"ContainerStarted","Data":"956a63f82493302ee001b6d544e5275ad2f4ff6098c7c34b255f25d5cb55a2c8"} Sep 10 13:21:16.995718 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:16.995694 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gtqlh_52fceedd-00f2-4943-ae47-dec51216d8e4/console-operator/0.log" Sep 10 13:21:16.995829 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:16.995735 2560 generic.go:356] "Generic (PLEG): container finished" podID="52fceedd-00f2-4943-ae47-dec51216d8e4" containerID="445d1d9740a6a3466cb1d32e10c589238a9c11e26728af44f34c427d48037ec4" exitCode=255 Sep 10 13:21:16.995829 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:16.995809 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-gtqlh" event={"ID":"52fceedd-00f2-4943-ae47-dec51216d8e4","Type":"ContainerDied","Data":"445d1d9740a6a3466cb1d32e10c589238a9c11e26728af44f34c427d48037ec4"} Sep 10 13:21:16.995990 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:16.995978 2560 scope.go:117] "RemoveContainer" containerID="445d1d9740a6a3466cb1d32e10c589238a9c11e26728af44f34c427d48037ec4" Sep 10 13:21:16.997802 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:16.997781 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4hmcg" event={"ID":"9c97ef99-3ac3-4334-aba7-f9de4a6cccf6","Type":"ContainerStarted","Data":"bb3dc221e8210715ee059bf26edbacb7757acd0bd9ccd447fa617cac0c271b57"} Sep 10 13:21:17.010756 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:17.010676 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-zgs9h" podStartSLOduration=2.132754569 podStartE2EDuration="8.010657394s" totalImagesPullingTime="5.877902825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:21:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:21:17.010492448 +0000 UTC m=+186.905209868" watchObservedRunningTime="2026-09-10 13:21:17.010657394 +0000 UTC m=+186.905374814" Sep 10 13:21:17.041356 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:17.041299 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4hmcg" podStartSLOduration=45.210470449 podStartE2EDuration="51.041278961s" totalImagesPullingTime="5.830808512s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:20:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:21:17.041069599 +0000 UTC m=+186.935787018" watchObservedRunningTime="2026-09-10 13:21:17.041278961 +0000 UTC m=+186.935996382" Sep 10 13:21:17.063071 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:17.062999 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-9zr8q" podStartSLOduration=45.146386714 podStartE2EDuration="51.062977056s" totalImagesPullingTime="5.916590342s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:20:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:21:17.062801162 +0000 UTC m=+186.957518581" watchObservedRunningTime="2026-09-10 13:21:17.062977056 +0000 UTC m=+186.957694467" Sep 10 13:21:17.084569 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:17.084511 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-482xw" podStartSLOduration=45.078563167 podStartE2EDuration="51.084491656s" totalImagesPullingTime="6.005928489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:20:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:21:17.084148358 +0000 UTC m=+186.978865777" watchObservedRunningTime="2026-09-10 13:21:17.084491656 +0000 UTC m=+186.979209078" Sep 10 13:21:17.107601 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:17.107545 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bdnkb" podStartSLOduration=45.339797017 podStartE2EDuration="51.107522909s" totalImagesPullingTime="5.767725892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:20:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:21:17.106156947 +0000 UTC m=+187.000874368" watchObservedRunningTime="2026-09-10 13:21:17.107522909 +0000 UTC m=+187.002240332" Sep 10 13:21:17.149264 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:17.149074 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-mnksz" podStartSLOduration=45.142494422 podStartE2EDuration="51.149049609s" totalImagesPullingTime="6.006555187s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:20:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:21:17.129732629 +0000 UTC m=+187.024450052" watchObservedRunningTime="2026-09-10 13:21:17.149049609 +0000 UTC m=+187.043767027" Sep 10 13:21:17.191970 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:17.191939 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:17.296211 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:17.296120 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:17.628783 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:17.628684 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e7fc744a-ed33-44a7-94f1-19bbd01b9c50-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-qw7g8\" (UID: \"e7fc744a-ed33-44a7-94f1-19bbd01b9c50\") " pod="openshift-network-console/networking-console-plugin-79977f9844-qw7g8" Sep 10 13:21:17.628783 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:17.628761 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4a051c0c-418c-439b-94e1-181c0a84e86b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-8k875\" (UID: \"4a051c0c-418c-439b-94e1-181c0a84e86b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8k875" Sep 10 13:21:17.629017 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:17.628798 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-service-ca-bundle\") pod \"router-default-5787c86768-n9xzb\" (UID: \"b19de8c5-85f7-4f5e-9011-8f38d0b80a69\") " pod="openshift-ingress/router-default-5787c86768-n9xzb" Sep 10 13:21:17.629017 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:17.628830 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/74a231ea-0f6d-4d1a-9e98-b54fa96a73bb-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-qkdhm\" (UID: \"74a231ea-0f6d-4d1a-9e98-b54fa96a73bb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qkdhm" Sep 10 13:21:17.629017 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:17.628864 2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 13:21:17.629017 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:17.628880 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-metrics-certs\") pod \"router-default-5787c86768-n9xzb\" (UID: \"b19de8c5-85f7-4f5e-9011-8f38d0b80a69\") " pod="openshift-ingress/router-default-5787c86768-n9xzb" Sep 10 13:21:17.629017 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:17.628916 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7fe46372-65c2-451c-9fba-11399f8c753d-metrics-tls\") pod \"dns-default-2tqpb\" (UID: \"7fe46372-65c2-451c-9fba-11399f8c753d\") " pod="openshift-dns/dns-default-2tqpb" Sep 10 13:21:17.629017 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:17.628943 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7fc744a-ed33-44a7-94f1-19bbd01b9c50-networking-console-plugin-cert podName:e7fc744a-ed33-44a7-94f1-19bbd01b9c50 nodeName:}" failed. No retries permitted until 2026-09-10 13:21:25.628921072 +0000 UTC m=+195.523638472 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e7fc744a-ed33-44a7-94f1-19bbd01b9c50-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-qw7g8" (UID: "e7fc744a-ed33-44a7-94f1-19bbd01b9c50") : secret "networking-console-plugin-cert" not found Sep 10 13:21:17.629017 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:17.628982 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3ef3618e-e54b-46bf-8347-69b8e72720fd-registry-tls\") pod \"image-registry-7d5587b5d-tzd9q\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" Sep 10 13:21:17.629017 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:17.629002 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 13:21:17.629437 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:17.629049 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fe46372-65c2-451c-9fba-11399f8c753d-metrics-tls podName:7fe46372-65c2-451c-9fba-11399f8c753d nodeName:}" failed. No retries permitted until 2026-09-10 13:21:25.629034164 +0000 UTC m=+195.523751566 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7fe46372-65c2-451c-9fba-11399f8c753d-metrics-tls") pod "dns-default-2tqpb" (UID: "7fe46372-65c2-451c-9fba-11399f8c753d") : secret "dns-default-metrics-tls" not found Sep 10 13:21:17.629437 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:17.629077 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 13:21:17.629437 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:17.629088 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d5587b5d-tzd9q: secret "image-registry-tls" not found Sep 10 13:21:17.629437 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:17.629099 2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 13:21:17.629437 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:17.629130 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3ef3618e-e54b-46bf-8347-69b8e72720fd-registry-tls podName:3ef3618e-e54b-46bf-8347-69b8e72720fd nodeName:}" failed. No retries permitted until 2026-09-10 13:21:25.629119484 +0000 UTC m=+195.523836884 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3ef3618e-e54b-46bf-8347-69b8e72720fd-registry-tls") pod "image-registry-7d5587b5d-tzd9q" (UID: "3ef3618e-e54b-46bf-8347-69b8e72720fd") : secret "image-registry-tls" not found Sep 10 13:21:17.629437 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:17.629146 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a051c0c-418c-439b-94e1-181c0a84e86b-cluster-monitoring-operator-tls podName:4a051c0c-418c-439b-94e1-181c0a84e86b nodeName:}" failed. No retries permitted until 2026-09-10 13:21:25.629137101 +0000 UTC m=+195.523854498 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4a051c0c-418c-439b-94e1-181c0a84e86b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-8k875" (UID: "4a051c0c-418c-439b-94e1-181c0a84e86b") : secret "cluster-monitoring-operator-tls" not found Sep 10 13:21:17.629437 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:17.629191 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-service-ca-bundle podName:b19de8c5-85f7-4f5e-9011-8f38d0b80a69 nodeName:}" failed. No retries permitted until 2026-09-10 13:21:25.629165522 +0000 UTC m=+195.523882921 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-service-ca-bundle") pod "router-default-5787c86768-n9xzb" (UID: "b19de8c5-85f7-4f5e-9011-8f38d0b80a69") : configmap references non-existent config key: service-ca.crt Sep 10 13:21:17.629437 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:17.629244 2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 13:21:17.629437 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:17.629273 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74a231ea-0f6d-4d1a-9e98-b54fa96a73bb-samples-operator-tls podName:74a231ea-0f6d-4d1a-9e98-b54fa96a73bb nodeName:}" failed. No retries permitted until 2026-09-10 13:21:25.629264115 +0000 UTC m=+195.523981519 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/74a231ea-0f6d-4d1a-9e98-b54fa96a73bb-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-qkdhm" (UID: "74a231ea-0f6d-4d1a-9e98-b54fa96a73bb") : secret "samples-operator-tls" not found Sep 10 13:21:17.629437 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:17.629322 2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 10 13:21:17.629437 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:17.629348 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-metrics-certs podName:b19de8c5-85f7-4f5e-9011-8f38d0b80a69 nodeName:}" failed. No retries permitted until 2026-09-10 13:21:25.629339268 +0000 UTC m=+195.524056667 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-metrics-certs") pod "router-default-5787c86768-n9xzb" (UID: "b19de8c5-85f7-4f5e-9011-8f38d0b80a69") : secret "router-metrics-certs-default" not found Sep 10 13:21:18.004065 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.004030 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gtqlh_52fceedd-00f2-4943-ae47-dec51216d8e4/console-operator/1.log" Sep 10 13:21:18.005132 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.005023 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gtqlh_52fceedd-00f2-4943-ae47-dec51216d8e4/console-operator/0.log" Sep 10 13:21:18.005132 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.005061 2560 generic.go:356] "Generic (PLEG): container finished" podID="52fceedd-00f2-4943-ae47-dec51216d8e4" containerID="601024dbc9c8470e3a2abcc385cd533b7d7be0d9dd79991f6b0c9306895ba1a6" exitCode=255 Sep 10 13:21:18.006921 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.006024 2560 scope.go:117] "RemoveContainer" containerID="601024dbc9c8470e3a2abcc385cd533b7d7be0d9dd79991f6b0c9306895ba1a6" Sep 10 13:21:18.006921 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:18.006177 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-gtqlh_openshift-console-operator(52fceedd-00f2-4943-ae47-dec51216d8e4)\"" pod="openshift-console-operator/console-operator-85b454874b-gtqlh" podUID="52fceedd-00f2-4943-ae47-dec51216d8e4" Sep 10 13:21:18.006921 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.006396 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-gtqlh" event={"ID":"52fceedd-00f2-4943-ae47-dec51216d8e4","Type":"ContainerDied","Data":"601024dbc9c8470e3a2abcc385cd533b7d7be0d9dd79991f6b0c9306895ba1a6"} Sep 10 13:21:18.006921 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.006458 2560 scope.go:117] "RemoveContainer" containerID="445d1d9740a6a3466cb1d32e10c589238a9c11e26728af44f34c427d48037ec4" Sep 10 13:21:18.096429 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.096377 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:18.099156 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.099127 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-9242g"] Sep 10 13:21:18.135721 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.135689 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9242g"] Sep 10 13:21:18.135874 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.135820 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9242g" Sep 10 13:21:18.138385 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.138359 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 10 13:21:18.138575 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.138562 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 10 13:21:18.138648 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.138629 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pbl4r\"" Sep 10 13:21:18.237251 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.237215 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vs8qm\" (UniqueName: \"kubernetes.io/projected/6f1d2f01-1377-4ca2-9b34-c70d033a0920-kube-api-access-vs8qm\") pod \"insights-runtime-extractor-9242g\" (UID: \"6f1d2f01-1377-4ca2-9b34-c70d033a0920\") " pod="openshift-insights/insights-runtime-extractor-9242g" Sep 10 13:21:18.237466 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.237256 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6f1d2f01-1377-4ca2-9b34-c70d033a0920-crio-socket\") pod \"insights-runtime-extractor-9242g\" (UID: \"6f1d2f01-1377-4ca2-9b34-c70d033a0920\") " pod="openshift-insights/insights-runtime-extractor-9242g" Sep 10 13:21:18.237466 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.237364 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f1d2f01-1377-4ca2-9b34-c70d033a0920-data-volume\") pod \"insights-runtime-extractor-9242g\" (UID: \"6f1d2f01-1377-4ca2-9b34-c70d033a0920\") " pod="openshift-insights/insights-runtime-extractor-9242g" Sep 10 13:21:18.237466 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.237414 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f1d2f01-1377-4ca2-9b34-c70d033a0920-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9242g\" (UID: \"6f1d2f01-1377-4ca2-9b34-c70d033a0920\") " pod="openshift-insights/insights-runtime-extractor-9242g" Sep 10 13:21:18.237588 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.237486 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f1d2f01-1377-4ca2-9b34-c70d033a0920-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9242g\" (UID: \"6f1d2f01-1377-4ca2-9b34-c70d033a0920\") " pod="openshift-insights/insights-runtime-extractor-9242g" Sep 10 13:21:18.261816 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.261722 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:18.338152 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.338114 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vs8qm\" (UniqueName: \"kubernetes.io/projected/6f1d2f01-1377-4ca2-9b34-c70d033a0920-kube-api-access-vs8qm\") pod \"insights-runtime-extractor-9242g\" (UID: \"6f1d2f01-1377-4ca2-9b34-c70d033a0920\") " pod="openshift-insights/insights-runtime-extractor-9242g" Sep 10 13:21:18.338152 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.338152 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6f1d2f01-1377-4ca2-9b34-c70d033a0920-crio-socket\") pod \"insights-runtime-extractor-9242g\" (UID: \"6f1d2f01-1377-4ca2-9b34-c70d033a0920\") " pod="openshift-insights/insights-runtime-extractor-9242g" Sep 10 13:21:18.338394 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.338180 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f1d2f01-1377-4ca2-9b34-c70d033a0920-data-volume\") pod \"insights-runtime-extractor-9242g\" (UID: \"6f1d2f01-1377-4ca2-9b34-c70d033a0920\") " pod="openshift-insights/insights-runtime-extractor-9242g" Sep 10 13:21:18.338394 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.338232 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f1d2f01-1377-4ca2-9b34-c70d033a0920-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9242g\" (UID: \"6f1d2f01-1377-4ca2-9b34-c70d033a0920\") " pod="openshift-insights/insights-runtime-extractor-9242g" Sep 10 13:21:18.338394 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.338263 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6f1d2f01-1377-4ca2-9b34-c70d033a0920-crio-socket\") pod \"insights-runtime-extractor-9242g\" (UID: \"6f1d2f01-1377-4ca2-9b34-c70d033a0920\") " pod="openshift-insights/insights-runtime-extractor-9242g" Sep 10 13:21:18.338394 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:18.338345 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 13:21:18.338631 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.338423 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f1d2f01-1377-4ca2-9b34-c70d033a0920-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9242g\" (UID: \"6f1d2f01-1377-4ca2-9b34-c70d033a0920\") " pod="openshift-insights/insights-runtime-extractor-9242g" Sep 10 13:21:18.338631 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:18.338438 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f1d2f01-1377-4ca2-9b34-c70d033a0920-insights-runtime-extractor-tls podName:6f1d2f01-1377-4ca2-9b34-c70d033a0920 nodeName:}" failed. No retries permitted until 2026-09-10 13:21:18.838396413 +0000 UTC m=+188.733113822 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6f1d2f01-1377-4ca2-9b34-c70d033a0920-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9242g" (UID: "6f1d2f01-1377-4ca2-9b34-c70d033a0920") : secret "insights-runtime-extractor-tls" not found Sep 10 13:21:18.338631 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:18.338471 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 13:21:18.338631 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:18.338508 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6f1d2f01-1377-4ca2-9b34-c70d033a0920-kube-rbac-proxy-cm podName:6f1d2f01-1377-4ca2-9b34-c70d033a0920 nodeName:}" failed. No retries permitted until 2026-09-10 13:21:18.838496327 +0000 UTC m=+188.733213726 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6f1d2f01-1377-4ca2-9b34-c70d033a0920-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9242g" (UID: "6f1d2f01-1377-4ca2-9b34-c70d033a0920") : configmap "kube-rbac-proxy" not found Sep 10 13:21:18.338631 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.338521 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f1d2f01-1377-4ca2-9b34-c70d033a0920-data-volume\") pod \"insights-runtime-extractor-9242g\" (UID: \"6f1d2f01-1377-4ca2-9b34-c70d033a0920\") " pod="openshift-insights/insights-runtime-extractor-9242g" Sep 10 13:21:18.348430 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.348389 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vs8qm\" (UniqueName: \"kubernetes.io/projected/6f1d2f01-1377-4ca2-9b34-c70d033a0920-kube-api-access-vs8qm\") pod \"insights-runtime-extractor-9242g\" (UID: \"6f1d2f01-1377-4ca2-9b34-c70d033a0920\") " pod="openshift-insights/insights-runtime-extractor-9242g" Sep 10 13:21:18.496001 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.495961 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:18.544808 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.544734 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-99m7r"] Sep 10 13:21:18.567578 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.567541 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-99m7r" Sep 10 13:21:18.570221 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.570194 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-dphm8\"" Sep 10 13:21:18.570221 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.570214 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Sep 10 13:21:18.570416 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.570195 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Sep 10 13:21:18.572176 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.572157 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-99m7r"] Sep 10 13:21:18.641001 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.640966 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prkgl\" (UniqueName: \"kubernetes.io/projected/d441e59a-93cc-4fd4-a3ad-df0831786472-kube-api-access-prkgl\") pod \"migrator-fdbb8fdff-99m7r\" (UID: \"d441e59a-93cc-4fd4-a3ad-df0831786472\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-99m7r" Sep 10 13:21:18.642009 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.641978 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:18.742240 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.742200 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prkgl\" (UniqueName: \"kubernetes.io/projected/d441e59a-93cc-4fd4-a3ad-df0831786472-kube-api-access-prkgl\") pod \"migrator-fdbb8fdff-99m7r\" (UID: \"d441e59a-93cc-4fd4-a3ad-df0831786472\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-99m7r" Sep 10 13:21:18.750950 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.750918 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:18.753964 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.753939 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prkgl\" (UniqueName: \"kubernetes.io/projected/d441e59a-93cc-4fd4-a3ad-df0831786472-kube-api-access-prkgl\") pod \"migrator-fdbb8fdff-99m7r\" (UID: \"d441e59a-93cc-4fd4-a3ad-df0831786472\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-99m7r" Sep 10 13:21:18.843084 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.842987 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f1d2f01-1377-4ca2-9b34-c70d033a0920-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9242g\" (UID: \"6f1d2f01-1377-4ca2-9b34-c70d033a0920\") " pod="openshift-insights/insights-runtime-extractor-9242g" Sep 10 13:21:18.843084 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.843053 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f1d2f01-1377-4ca2-9b34-c70d033a0920-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9242g\" (UID: \"6f1d2f01-1377-4ca2-9b34-c70d033a0920\") " pod="openshift-insights/insights-runtime-extractor-9242g" Sep 10 13:21:18.843262 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:18.843157 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 13:21:18.843262 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:18.843190 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 13:21:18.843262 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:18.843237 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f1d2f01-1377-4ca2-9b34-c70d033a0920-insights-runtime-extractor-tls podName:6f1d2f01-1377-4ca2-9b34-c70d033a0920 nodeName:}" failed. No retries permitted until 2026-09-10 13:21:19.843220351 +0000 UTC m=+189.737937757 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6f1d2f01-1377-4ca2-9b34-c70d033a0920-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9242g" (UID: "6f1d2f01-1377-4ca2-9b34-c70d033a0920") : secret "insights-runtime-extractor-tls" not found Sep 10 13:21:18.843262 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:18.843260 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6f1d2f01-1377-4ca2-9b34-c70d033a0920-kube-rbac-proxy-cm podName:6f1d2f01-1377-4ca2-9b34-c70d033a0920 nodeName:}" failed. No retries permitted until 2026-09-10 13:21:19.843246094 +0000 UTC m=+189.737963493 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6f1d2f01-1377-4ca2-9b34-c70d033a0920-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9242g" (UID: "6f1d2f01-1377-4ca2-9b34-c70d033a0920") : configmap "kube-rbac-proxy" not found Sep 10 13:21:18.878187 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.878161 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-99m7r" Sep 10 13:21:18.942342 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:18.942304 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:19.010256 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:19.010230 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gtqlh_52fceedd-00f2-4943-ae47-dec51216d8e4/console-operator/1.log" Sep 10 13:21:19.010637 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:19.010583 2560 scope.go:117] "RemoveContainer" containerID="601024dbc9c8470e3a2abcc385cd533b7d7be0d9dd79991f6b0c9306895ba1a6" Sep 10 13:21:19.010770 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:19.010751 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-gtqlh_openshift-console-operator(52fceedd-00f2-4943-ae47-dec51216d8e4)\"" pod="openshift-console-operator/console-operator-85b454874b-gtqlh" podUID="52fceedd-00f2-4943-ae47-dec51216d8e4" Sep 10 13:21:19.143487 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:19.143385 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:19.173242 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:19.173046 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-99m7r"] Sep 10 13:21:19.175639 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:21:19.175614 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd441e59a_93cc_4fd4_a3ad_df0831786472.slice/crio-889e99b83062116b9b507912e2da4bb64c6e4bab32570476704d34b23b1b0170 WatchSource:0}: Error finding container 889e99b83062116b9b507912e2da4bb64c6e4bab32570476704d34b23b1b0170: Status 404 returned error can't find the container with id 889e99b83062116b9b507912e2da4bb64c6e4bab32570476704d34b23b1b0170 Sep 10 13:21:19.204754 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:19.204729 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jv5qh_75c33310-0b35-4c84-be13-75a0d65d95e9/dns-node-resolver/0.log" Sep 10 13:21:19.851629 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:19.851600 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f1d2f01-1377-4ca2-9b34-c70d033a0920-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9242g\" (UID: \"6f1d2f01-1377-4ca2-9b34-c70d033a0920\") " pod="openshift-insights/insights-runtime-extractor-9242g" Sep 10 13:21:19.851829 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:19.851731 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 13:21:19.851829 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:19.851744 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f1d2f01-1377-4ca2-9b34-c70d033a0920-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9242g\" (UID: \"6f1d2f01-1377-4ca2-9b34-c70d033a0920\") " pod="openshift-insights/insights-runtime-extractor-9242g" Sep 10 13:21:19.851829 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:19.851802 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6f1d2f01-1377-4ca2-9b34-c70d033a0920-kube-rbac-proxy-cm podName:6f1d2f01-1377-4ca2-9b34-c70d033a0920 nodeName:}" failed. No retries permitted until 2026-09-10 13:21:21.851781102 +0000 UTC m=+191.746498512 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6f1d2f01-1377-4ca2-9b34-c70d033a0920-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9242g" (UID: "6f1d2f01-1377-4ca2-9b34-c70d033a0920") : configmap "kube-rbac-proxy" not found Sep 10 13:21:19.851964 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:19.851840 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 13:21:19.851964 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:19.851894 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f1d2f01-1377-4ca2-9b34-c70d033a0920-insights-runtime-extractor-tls podName:6f1d2f01-1377-4ca2-9b34-c70d033a0920 nodeName:}" failed. No retries permitted until 2026-09-10 13:21:21.851877229 +0000 UTC m=+191.746594640 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6f1d2f01-1377-4ca2-9b34-c70d033a0920-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9242g" (UID: "6f1d2f01-1377-4ca2-9b34-c70d033a0920") : secret "insights-runtime-extractor-tls" not found Sep 10 13:21:20.015052 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:20.015013 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-99m7r" event={"ID":"d441e59a-93cc-4fd4-a3ad-df0831786472","Type":"ContainerStarted","Data":"889e99b83062116b9b507912e2da4bb64c6e4bab32570476704d34b23b1b0170"} Sep 10 13:21:20.130702 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:20.130620 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-gtqlh" Sep 10 13:21:20.130702 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:20.130685 2560 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-gtqlh" Sep 10 13:21:20.131025 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:20.131012 2560 scope.go:117] "RemoveContainer" containerID="601024dbc9c8470e3a2abcc385cd533b7d7be0d9dd79991f6b0c9306895ba1a6" Sep 10 13:21:20.131190 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:20.131171 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-gtqlh_openshift-console-operator(52fceedd-00f2-4943-ae47-dec51216d8e4)\"" pod="openshift-console-operator/console-operator-85b454874b-gtqlh" podUID="52fceedd-00f2-4943-ae47-dec51216d8e4" Sep 10 13:21:20.203456 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:20.203427 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ps7jp_611016d1-55f4-4537-b5cb-2bd50af1d7fb/node-ca/0.log" Sep 10 13:21:20.255548 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:20.255511 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-75d99c484-hvxx4"] Sep 10 13:21:20.270917 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:20.270880 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-hvxx4"] Sep 10 13:21:20.271094 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:20.270950 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-hvxx4" Sep 10 13:21:20.274765 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:20.274739 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Sep 10 13:21:20.274917 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:20.274854 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Sep 10 13:21:20.275175 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:20.275151 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Sep 10 13:21:20.275175 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:20.275174 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Sep 10 13:21:20.275753 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:20.275736 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-krz44\"" Sep 10 13:21:20.293057 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:20.293019 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:20.356425 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:20.356374 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/bcec7345-021e-4fc8-85a4-3a6cbbc022d8-signing-key\") pod \"service-ca-75d99c484-hvxx4\" (UID: \"bcec7345-021e-4fc8-85a4-3a6cbbc022d8\") " pod="openshift-service-ca/service-ca-75d99c484-hvxx4" Sep 10 13:21:20.356644 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:20.356457 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/bcec7345-021e-4fc8-85a4-3a6cbbc022d8-signing-cabundle\") pod \"service-ca-75d99c484-hvxx4\" (UID: \"bcec7345-021e-4fc8-85a4-3a6cbbc022d8\") " pod="openshift-service-ca/service-ca-75d99c484-hvxx4" Sep 10 13:21:20.356644 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:20.356545 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fb57f\" (UniqueName: \"kubernetes.io/projected/bcec7345-021e-4fc8-85a4-3a6cbbc022d8-kube-api-access-fb57f\") pod \"service-ca-75d99c484-hvxx4\" (UID: \"bcec7345-021e-4fc8-85a4-3a6cbbc022d8\") " pod="openshift-service-ca/service-ca-75d99c484-hvxx4" Sep 10 13:21:20.382502 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:20.382391 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:20.457783 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:20.457743 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fb57f\" (UniqueName: \"kubernetes.io/projected/bcec7345-021e-4fc8-85a4-3a6cbbc022d8-kube-api-access-fb57f\") pod \"service-ca-75d99c484-hvxx4\" (UID: \"bcec7345-021e-4fc8-85a4-3a6cbbc022d8\") " pod="openshift-service-ca/service-ca-75d99c484-hvxx4" Sep 10 13:21:20.457983 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:20.457923 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/bcec7345-021e-4fc8-85a4-3a6cbbc022d8-signing-key\") pod \"service-ca-75d99c484-hvxx4\" (UID: \"bcec7345-021e-4fc8-85a4-3a6cbbc022d8\") " pod="openshift-service-ca/service-ca-75d99c484-hvxx4" Sep 10 13:21:20.458047 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:20.458000 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/bcec7345-021e-4fc8-85a4-3a6cbbc022d8-signing-cabundle\") pod \"service-ca-75d99c484-hvxx4\" (UID: \"bcec7345-021e-4fc8-85a4-3a6cbbc022d8\") " pod="openshift-service-ca/service-ca-75d99c484-hvxx4" Sep 10 13:21:20.458876 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:20.458850 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/bcec7345-021e-4fc8-85a4-3a6cbbc022d8-signing-cabundle\") pod \"service-ca-75d99c484-hvxx4\" (UID: \"bcec7345-021e-4fc8-85a4-3a6cbbc022d8\") " pod="openshift-service-ca/service-ca-75d99c484-hvxx4" Sep 10 13:21:20.461082 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:20.461061 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/bcec7345-021e-4fc8-85a4-3a6cbbc022d8-signing-key\") pod \"service-ca-75d99c484-hvxx4\" (UID: \"bcec7345-021e-4fc8-85a4-3a6cbbc022d8\") " pod="openshift-service-ca/service-ca-75d99c484-hvxx4" Sep 10 13:21:20.467169 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:20.467148 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fb57f\" (UniqueName: \"kubernetes.io/projected/bcec7345-021e-4fc8-85a4-3a6cbbc022d8-kube-api-access-fb57f\") pod \"service-ca-75d99c484-hvxx4\" (UID: \"bcec7345-021e-4fc8-85a4-3a6cbbc022d8\") " pod="openshift-service-ca/service-ca-75d99c484-hvxx4" Sep 10 13:21:20.582309 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:20.582284 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-hvxx4" Sep 10 13:21:20.604352 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:20.604329 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zgs9h_227a2714-52ab-48f6-904b-9e997ed5f3a3/serve-healthcheck-canary/0.log" Sep 10 13:21:20.829149 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:20.829118 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:20.862464 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:20.862435 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-hvxx4"] Sep 10 13:21:20.865677 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:21:20.865651 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcec7345_021e_4fc8_85a4_3a6cbbc022d8.slice/crio-6877d8fa2b744abd7a21bc03b4f8260643cdb0e32c2984eefdbef6b433026b26 WatchSource:0}: Error finding container 6877d8fa2b744abd7a21bc03b4f8260643cdb0e32c2984eefdbef6b433026b26: Status 404 returned error can't find the container with id 6877d8fa2b744abd7a21bc03b4f8260643cdb0e32c2984eefdbef6b433026b26 Sep 10 13:21:20.936636 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:20.936594 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:21.020469 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:21.020432 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-hvxx4" event={"ID":"bcec7345-021e-4fc8-85a4-3a6cbbc022d8","Type":"ContainerStarted","Data":"447be0c7117b1563b394740ddc67786ae9b7c7dc759736d8398b2ca2c7f7a7ba"} Sep 10 13:21:21.020943 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:21.020489 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-hvxx4" event={"ID":"bcec7345-021e-4fc8-85a4-3a6cbbc022d8","Type":"ContainerStarted","Data":"6877d8fa2b744abd7a21bc03b4f8260643cdb0e32c2984eefdbef6b433026b26"} Sep 10 13:21:21.022367 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:21.022340 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-99m7r" event={"ID":"d441e59a-93cc-4fd4-a3ad-df0831786472","Type":"ContainerStarted","Data":"299b878e6cbd9c72d60c3122f7b54e00b9cb8a209d62c194e52d33c79b5b2fb5"} Sep 10 13:21:21.022490 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:21.022385 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-99m7r" event={"ID":"d441e59a-93cc-4fd4-a3ad-df0831786472","Type":"ContainerStarted","Data":"9a5049e28ff03c9bd4ee330c5916df2761f41a4e14d595e4d720b074eb225d36"} Sep 10 13:21:21.034813 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:21.034787 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:21.047794 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:21.047750 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-75d99c484-hvxx4" podStartSLOduration=1.047735279 podStartE2EDuration="1.047735279s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:21:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:21:21.046924276 +0000 UTC m=+190.941641695" watchObservedRunningTime="2026-09-10 13:21:21.047735279 +0000 UTC m=+190.942452698" Sep 10 13:21:21.460372 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:21.460317 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:21.870917 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:21.870828 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f1d2f01-1377-4ca2-9b34-c70d033a0920-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9242g\" (UID: \"6f1d2f01-1377-4ca2-9b34-c70d033a0920\") " pod="openshift-insights/insights-runtime-extractor-9242g" Sep 10 13:21:21.870917 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:21.870886 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f1d2f01-1377-4ca2-9b34-c70d033a0920-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9242g\" (UID: \"6f1d2f01-1377-4ca2-9b34-c70d033a0920\") " pod="openshift-insights/insights-runtime-extractor-9242g" Sep 10 13:21:21.871127 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:21.870951 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 13:21:21.871127 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:21.871038 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6f1d2f01-1377-4ca2-9b34-c70d033a0920-kube-rbac-proxy-cm podName:6f1d2f01-1377-4ca2-9b34-c70d033a0920 nodeName:}" failed. No retries permitted until 2026-09-10 13:21:25.871019441 +0000 UTC m=+195.765736837 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6f1d2f01-1377-4ca2-9b34-c70d033a0920-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9242g" (UID: "6f1d2f01-1377-4ca2-9b34-c70d033a0920") : configmap "kube-rbac-proxy" not found Sep 10 13:21:21.871127 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:21.871099 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 13:21:21.871247 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:21.871144 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f1d2f01-1377-4ca2-9b34-c70d033a0920-insights-runtime-extractor-tls podName:6f1d2f01-1377-4ca2-9b34-c70d033a0920 nodeName:}" failed. No retries permitted until 2026-09-10 13:21:25.871134555 +0000 UTC m=+195.765851967 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6f1d2f01-1377-4ca2-9b34-c70d033a0920-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9242g" (UID: "6f1d2f01-1377-4ca2-9b34-c70d033a0920") : secret "insights-runtime-extractor-tls" not found Sep 10 13:21:22.843886 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:22.843849 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:22.948841 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:22.948810 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:23.055958 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:23.055921 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:23.188710 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:23.188674 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:23.368229 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:23.368177 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:23.478856 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:23.478724 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:23.579557 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:23.579518 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:24.514135 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:24.514102 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:24.929463 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:24.929426 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:25.103565 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:25.103527 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:25.306131 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:25.306035 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:25.500453 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:25.500420 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:25.706758 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:25.706722 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3ef3618e-e54b-46bf-8347-69b8e72720fd-registry-tls\") pod \"image-registry-7d5587b5d-tzd9q\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" Sep 10 13:21:25.707216 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:25.706815 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e7fc744a-ed33-44a7-94f1-19bbd01b9c50-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-qw7g8\" (UID: \"e7fc744a-ed33-44a7-94f1-19bbd01b9c50\") " pod="openshift-network-console/networking-console-plugin-79977f9844-qw7g8" Sep 10 13:21:25.707216 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:25.706874 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 13:21:25.707216 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:25.706892 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d5587b5d-tzd9q: secret "image-registry-tls" not found Sep 10 13:21:25.707216 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:25.706943 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3ef3618e-e54b-46bf-8347-69b8e72720fd-registry-tls podName:3ef3618e-e54b-46bf-8347-69b8e72720fd nodeName:}" failed. No retries permitted until 2026-09-10 13:21:41.70692877 +0000 UTC m=+211.601646165 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3ef3618e-e54b-46bf-8347-69b8e72720fd-registry-tls") pod "image-registry-7d5587b5d-tzd9q" (UID: "3ef3618e-e54b-46bf-8347-69b8e72720fd") : secret "image-registry-tls" not found Sep 10 13:21:25.707216 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:25.706950 2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 13:21:25.707216 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:25.706874 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4a051c0c-418c-439b-94e1-181c0a84e86b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-8k875\" (UID: \"4a051c0c-418c-439b-94e1-181c0a84e86b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8k875" Sep 10 13:21:25.707216 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:25.706950 2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 13:21:25.707216 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:25.707005 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a051c0c-418c-439b-94e1-181c0a84e86b-cluster-monitoring-operator-tls podName:4a051c0c-418c-439b-94e1-181c0a84e86b nodeName:}" failed. No retries permitted until 2026-09-10 13:21:41.706986402 +0000 UTC m=+211.601703798 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4a051c0c-418c-439b-94e1-181c0a84e86b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-8k875" (UID: "4a051c0c-418c-439b-94e1-181c0a84e86b") : secret "cluster-monitoring-operator-tls" not found Sep 10 13:21:25.707216 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:25.707024 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-service-ca-bundle\") pod \"router-default-5787c86768-n9xzb\" (UID: \"b19de8c5-85f7-4f5e-9011-8f38d0b80a69\") " pod="openshift-ingress/router-default-5787c86768-n9xzb" Sep 10 13:21:25.707216 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:25.707053 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7fc744a-ed33-44a7-94f1-19bbd01b9c50-networking-console-plugin-cert podName:e7fc744a-ed33-44a7-94f1-19bbd01b9c50 nodeName:}" failed. No retries permitted until 2026-09-10 13:21:41.707027372 +0000 UTC m=+211.601744793 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e7fc744a-ed33-44a7-94f1-19bbd01b9c50-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-qw7g8" (UID: "e7fc744a-ed33-44a7-94f1-19bbd01b9c50") : secret "networking-console-plugin-cert" not found Sep 10 13:21:25.707216 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:25.707088 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/74a231ea-0f6d-4d1a-9e98-b54fa96a73bb-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-qkdhm\" (UID: \"74a231ea-0f6d-4d1a-9e98-b54fa96a73bb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qkdhm" Sep 10 13:21:25.707216 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:25.707137 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-service-ca-bundle podName:b19de8c5-85f7-4f5e-9011-8f38d0b80a69 nodeName:}" failed. No retries permitted until 2026-09-10 13:21:41.707123877 +0000 UTC m=+211.601841273 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-service-ca-bundle") pod "router-default-5787c86768-n9xzb" (UID: "b19de8c5-85f7-4f5e-9011-8f38d0b80a69") : configmap references non-existent config key: service-ca.crt Sep 10 13:21:25.707216 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:25.707171 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-metrics-certs\") pod \"router-default-5787c86768-n9xzb\" (UID: \"b19de8c5-85f7-4f5e-9011-8f38d0b80a69\") " pod="openshift-ingress/router-default-5787c86768-n9xzb" Sep 10 13:21:25.707216 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:25.707213 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7fe46372-65c2-451c-9fba-11399f8c753d-metrics-tls\") pod \"dns-default-2tqpb\" (UID: \"7fe46372-65c2-451c-9fba-11399f8c753d\") " pod="openshift-dns/dns-default-2tqpb" Sep 10 13:21:25.707766 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:25.707274 2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 10 13:21:25.707766 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:25.707297 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 13:21:25.707766 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:25.707304 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-metrics-certs podName:b19de8c5-85f7-4f5e-9011-8f38d0b80a69 nodeName:}" failed. No retries permitted until 2026-09-10 13:21:41.707295592 +0000 UTC m=+211.602012988 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-metrics-certs") pod "router-default-5787c86768-n9xzb" (UID: "b19de8c5-85f7-4f5e-9011-8f38d0b80a69") : secret "router-metrics-certs-default" not found Sep 10 13:21:25.707766 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:25.707324 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fe46372-65c2-451c-9fba-11399f8c753d-metrics-tls podName:7fe46372-65c2-451c-9fba-11399f8c753d nodeName:}" failed. No retries permitted until 2026-09-10 13:21:41.707316408 +0000 UTC m=+211.602033805 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7fe46372-65c2-451c-9fba-11399f8c753d-metrics-tls") pod "dns-default-2tqpb" (UID: "7fe46372-65c2-451c-9fba-11399f8c753d") : secret "dns-default-metrics-tls" not found Sep 10 13:21:25.709418 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:25.709380 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/74a231ea-0f6d-4d1a-9e98-b54fa96a73bb-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-qkdhm\" (UID: \"74a231ea-0f6d-4d1a-9e98-b54fa96a73bb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qkdhm" Sep 10 13:21:25.719345 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:25.719324 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-j8q7v\"" Sep 10 13:21:25.727861 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:25.727840 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qkdhm" Sep 10 13:21:25.909050 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:25.909022 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f1d2f01-1377-4ca2-9b34-c70d033a0920-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9242g\" (UID: \"6f1d2f01-1377-4ca2-9b34-c70d033a0920\") " pod="openshift-insights/insights-runtime-extractor-9242g" Sep 10 13:21:25.909220 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:25.909074 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f1d2f01-1377-4ca2-9b34-c70d033a0920-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9242g\" (UID: \"6f1d2f01-1377-4ca2-9b34-c70d033a0920\") " pod="openshift-insights/insights-runtime-extractor-9242g" Sep 10 13:21:25.909220 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:25.909152 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 13:21:25.909220 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:25.909192 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 13:21:25.909220 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:25.909210 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6f1d2f01-1377-4ca2-9b34-c70d033a0920-kube-rbac-proxy-cm podName:6f1d2f01-1377-4ca2-9b34-c70d033a0920 nodeName:}" failed. No retries permitted until 2026-09-10 13:21:33.909192006 +0000 UTC m=+203.803909423 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6f1d2f01-1377-4ca2-9b34-c70d033a0920-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9242g" (UID: "6f1d2f01-1377-4ca2-9b34-c70d033a0920") : configmap "kube-rbac-proxy" not found Sep 10 13:21:25.909385 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:25.909269 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f1d2f01-1377-4ca2-9b34-c70d033a0920-insights-runtime-extractor-tls podName:6f1d2f01-1377-4ca2-9b34-c70d033a0920 nodeName:}" failed. No retries permitted until 2026-09-10 13:21:33.909240393 +0000 UTC m=+203.803957806 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6f1d2f01-1377-4ca2-9b34-c70d033a0920-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9242g" (UID: "6f1d2f01-1377-4ca2-9b34-c70d033a0920") : secret "insights-runtime-extractor-tls" not found Sep 10 13:21:25.995794 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:25.995702 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-99m7r" podStartSLOduration=6.37859238 podStartE2EDuration="7.995682022s" totalImagesPullingTime="1.617089642s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:21:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:21:21.07360598 +0000 UTC m=+190.968323398" watchObservedRunningTime="2026-09-10 13:21:25.995682022 +0000 UTC m=+195.890399440" Sep 10 13:21:25.996514 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:25.996495 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qkdhm"] Sep 10 13:21:26.042362 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:26.042335 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qkdhm" event={"ID":"74a231ea-0f6d-4d1a-9e98-b54fa96a73bb","Type":"ContainerStarted","Data":"d4142db28cf29640289b6bce3455eb33d41e05fb46d24e93a59cf1f2714297dc"} Sep 10 13:21:26.495961 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:26.495917 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:26.625849 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:26.625814 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:26.772623 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:26.772541 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:26.969947 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:26.969906 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:27.106502 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:27.106413 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:27.302518 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:27.302464 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:27.390464 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:27.390375 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:27.504448 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:27.504389 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:27.578981 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:27.578947 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:27.924616 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:27.924592 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-n86g9" Sep 10 13:21:28.051807 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:28.051720 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qkdhm" event={"ID":"74a231ea-0f6d-4d1a-9e98-b54fa96a73bb","Type":"ContainerStarted","Data":"5ae0df101e76a0de68d10c1a5bd2794477fa270fa504e26cfe0562bcd7009d1a"} Sep 10 13:21:28.051983 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:28.051815 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qkdhm" event={"ID":"74a231ea-0f6d-4d1a-9e98-b54fa96a73bb","Type":"ContainerStarted","Data":"4b4df35faf8766884784b539daaa3684e08b694cfb5f9099bb8ba5a21c3ae2f4"} Sep 10 13:21:28.068078 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:28.067939 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-qkdhm" podStartSLOduration=60.353439696 podStartE2EDuration="1m2.067923132s" totalImagesPullingTime="1.714483436s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:20:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:21:28.067639136 +0000 UTC m=+197.962356554" watchObservedRunningTime="2026-09-10 13:21:28.067923132 +0000 UTC m=+197.962640553" Sep 10 13:21:28.715794 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:28.715755 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:28.917269 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:28.917227 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:29.108933 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:29.108838 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:29.260266 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:29.260236 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:29.406963 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:29.406867 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:29.507804 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:29.507751 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:29.605710 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:29.605675 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:30.728928 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:30.728886 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:30.953733 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:30.953698 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:31.054673 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:31.054598 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/117705e7-b9e7-47ee-a474-d7f74afb23b6-metrics-certs\") pod \"network-metrics-daemon-vrbmp\" (UID: \"117705e7-b9e7-47ee-a474-d7f74afb23b6\") " pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:21:31.057798 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:31.057780 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 13:21:31.067122 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:31.067101 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/117705e7-b9e7-47ee-a474-d7f74afb23b6-metrics-certs\") pod \"network-metrics-daemon-vrbmp\" (UID: \"117705e7-b9e7-47ee-a474-d7f74afb23b6\") " pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:21:31.119030 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:31.118994 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:31.155949 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:31.155916 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28v55\" (UniqueName: \"kubernetes.io/projected/3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597-kube-api-access-28v55\") pod \"network-check-target-g64cr\" (UID: \"3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597\") " pod="openshift-network-diagnostics/network-check-target-g64cr" Sep 10 13:21:31.158452 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:31.158428 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28v55\" (UniqueName: \"kubernetes.io/projected/3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597-kube-api-access-28v55\") pod \"network-check-target-g64cr\" (UID: \"3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597\") " pod="openshift-network-diagnostics/network-check-target-g64cr" Sep 10 13:21:31.169773 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:31.169747 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-67hjc\"" Sep 10 13:21:31.176824 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:31.176803 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g64cr" Sep 10 13:21:31.182916 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:31.182897 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4sjdq\"" Sep 10 13:21:31.187090 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:31.187059 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vrbmp" Sep 10 13:21:31.327351 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:31.327262 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:31.358537 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:31.358508 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/faa8bfdc-9c79-4c12-8ec9-3cd8639d0793-original-pull-secret\") pod \"global-pull-secret-syncer-v2hld\" (UID: \"faa8bfdc-9c79-4c12-8ec9-3cd8639d0793\") " pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:21:31.361628 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:31.361610 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 13:21:31.371298 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:31.371274 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/faa8bfdc-9c79-4c12-8ec9-3cd8639d0793-original-pull-secret\") pod \"global-pull-secret-syncer-v2hld\" (UID: \"faa8bfdc-9c79-4c12-8ec9-3cd8639d0793\") " pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:21:31.462498 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:31.462476 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-g64cr"] Sep 10 13:21:31.464744 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:21:31.464713 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d042a95_2f2b_4f81_b4f1_a7d8aaaa1597.slice/crio-762f10e3ecb9f00f4e04fd629a4a2470e27c196dc3aa65ea588a1765ac1753b9 WatchSource:0}: Error finding container 762f10e3ecb9f00f4e04fd629a4a2470e27c196dc3aa65ea588a1765ac1753b9: Status 404 returned error can't find the container with id 762f10e3ecb9f00f4e04fd629a4a2470e27c196dc3aa65ea588a1765ac1753b9 Sep 10 13:21:31.471588 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:31.471529 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v2hld" Sep 10 13:21:31.477491 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:31.477469 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vrbmp"] Sep 10 13:21:31.480874 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:21:31.480852 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod117705e7_b9e7_47ee_a474_d7f74afb23b6.slice/crio-56aeb0276f7a773c86e6e528727a8a0b83077dace65b793da476cbb9e60f9d7b WatchSource:0}: Error finding container 56aeb0276f7a773c86e6e528727a8a0b83077dace65b793da476cbb9e60f9d7b: Status 404 returned error can't find the container with id 56aeb0276f7a773c86e6e528727a8a0b83077dace65b793da476cbb9e60f9d7b Sep 10 13:21:31.525112 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:31.523903 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:31.745954 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:31.745923 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-v2hld"] Sep 10 13:21:31.748843 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:21:31.748820 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaa8bfdc_9c79_4c12_8ec9_3cd8639d0793.slice/crio-a92525da787d6591bbba598a47d29a1136b0e70f2129daf8e12e30e13a7fa31c WatchSource:0}: Error finding container a92525da787d6591bbba598a47d29a1136b0e70f2129daf8e12e30e13a7fa31c: Status 404 returned error can't find the container with id a92525da787d6591bbba598a47d29a1136b0e70f2129daf8e12e30e13a7fa31c Sep 10 13:21:32.067464 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:32.067376 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-v2hld" event={"ID":"faa8bfdc-9c79-4c12-8ec9-3cd8639d0793","Type":"ContainerStarted","Data":"a92525da787d6591bbba598a47d29a1136b0e70f2129daf8e12e30e13a7fa31c"} Sep 10 13:21:32.068497 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:32.068468 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vrbmp" event={"ID":"117705e7-b9e7-47ee-a474-d7f74afb23b6","Type":"ContainerStarted","Data":"56aeb0276f7a773c86e6e528727a8a0b83077dace65b793da476cbb9e60f9d7b"} Sep 10 13:21:32.069939 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:32.069917 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-g64cr" event={"ID":"3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597","Type":"ContainerStarted","Data":"e44cb5bdfad8e800180f22c3c77fc8f32a821fa0662e66c2da60d5498c1c6044"} Sep 10 13:21:32.070052 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:32.069953 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-g64cr" event={"ID":"3d042a95-2f2b-4f81-b4f1-a7d8aaaa1597","Type":"ContainerStarted","Data":"762f10e3ecb9f00f4e04fd629a4a2470e27c196dc3aa65ea588a1765ac1753b9"} Sep 10 13:21:32.070121 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:32.070065 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-g64cr" Sep 10 13:21:32.095819 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:32.095762 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-g64cr" podStartSLOduration=65.095745514 podStartE2EDuration="1m5.095745514s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:20:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:21:32.093759131 +0000 UTC m=+201.988476549" watchObservedRunningTime="2026-09-10 13:21:32.095745514 +0000 UTC m=+201.990462931" Sep 10 13:21:32.505046 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:32.505012 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:32.606759 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:32.606722 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:32.711315 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:32.711281 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:32.944050 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:32.943105 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:33.076102 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:33.076016 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vrbmp" event={"ID":"117705e7-b9e7-47ee-a474-d7f74afb23b6","Type":"ContainerStarted","Data":"21af6a9124989e122b4ff4eaeb8c206b43c772bfb5ea31691bf6e0bcbbce16de"} Sep 10 13:21:33.076102 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:33.076088 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vrbmp" event={"ID":"117705e7-b9e7-47ee-a474-d7f74afb23b6","Type":"ContainerStarted","Data":"12c66ab23a0965241932ca03dfb2a32c3ac8b59479018d9de13a00763f2a6c5d"} Sep 10 13:21:33.101206 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:33.101126 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-vrbmp" podStartSLOduration=64.769681969 podStartE2EDuration="1m6.101103908s" totalImagesPullingTime="1.331421939s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:20:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:21:33.100051907 +0000 UTC m=+202.994769328" watchObservedRunningTime="2026-09-10 13:21:33.101103908 +0000 UTC m=+202.995821327" Sep 10 13:21:33.133864 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:33.133824 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:33.508731 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:33.508695 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:33.557261 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:33.557227 2560 scope.go:117] "RemoveContainer" containerID="601024dbc9c8470e3a2abcc385cd533b7d7be0d9dd79991f6b0c9306895ba1a6" Sep 10 13:21:33.628969 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:33.628932 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:33.981930 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:33.981892 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f1d2f01-1377-4ca2-9b34-c70d033a0920-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9242g\" (UID: \"6f1d2f01-1377-4ca2-9b34-c70d033a0920\") " pod="openshift-insights/insights-runtime-extractor-9242g" Sep 10 13:21:33.982381 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:33.981967 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f1d2f01-1377-4ca2-9b34-c70d033a0920-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9242g\" (UID: \"6f1d2f01-1377-4ca2-9b34-c70d033a0920\") " pod="openshift-insights/insights-runtime-extractor-9242g" Sep 10 13:21:33.982635 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:33.982611 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f1d2f01-1377-4ca2-9b34-c70d033a0920-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9242g\" (UID: \"6f1d2f01-1377-4ca2-9b34-c70d033a0920\") " pod="openshift-insights/insights-runtime-extractor-9242g" Sep 10 13:21:33.984504 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:33.984478 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f1d2f01-1377-4ca2-9b34-c70d033a0920-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9242g\" (UID: \"6f1d2f01-1377-4ca2-9b34-c70d033a0920\") " pod="openshift-insights/insights-runtime-extractor-9242g" Sep 10 13:21:34.045644 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:34.045614 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9242g" Sep 10 13:21:34.082300 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:34.082269 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gtqlh_52fceedd-00f2-4943-ae47-dec51216d8e4/console-operator/2.log" Sep 10 13:21:34.082960 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:34.082940 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gtqlh_52fceedd-00f2-4943-ae47-dec51216d8e4/console-operator/1.log" Sep 10 13:21:34.083089 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:34.082982 2560 generic.go:356] "Generic (PLEG): container finished" podID="52fceedd-00f2-4943-ae47-dec51216d8e4" containerID="708b62fe219deb467b6409953de1babbae619f2b46f7c84e0f31bd6c97097db9" exitCode=255 Sep 10 13:21:34.083272 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:34.083249 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-gtqlh" event={"ID":"52fceedd-00f2-4943-ae47-dec51216d8e4","Type":"ContainerDied","Data":"708b62fe219deb467b6409953de1babbae619f2b46f7c84e0f31bd6c97097db9"} Sep 10 13:21:34.083328 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:34.083309 2560 scope.go:117] "RemoveContainer" containerID="601024dbc9c8470e3a2abcc385cd533b7d7be0d9dd79991f6b0c9306895ba1a6" Sep 10 13:21:34.083735 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:34.083711 2560 scope.go:117] "RemoveContainer" containerID="708b62fe219deb467b6409953de1babbae619f2b46f7c84e0f31bd6c97097db9" Sep 10 13:21:34.083959 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:34.083938 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-gtqlh_openshift-console-operator(52fceedd-00f2-4943-ae47-dec51216d8e4)\"" pod="openshift-console-operator/console-operator-85b454874b-gtqlh" podUID="52fceedd-00f2-4943-ae47-dec51216d8e4" Sep 10 13:21:34.262731 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:34.262642 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:34.329653 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:34.329617 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9242g"] Sep 10 13:21:34.333794 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:21:34.333764 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f1d2f01_1377_4ca2_9b34_c70d033a0920.slice/crio-4bc070913a5d3b2f99f78b1195cc86f758d1f995d701a7af2f2a9a2f5f18a0c3 WatchSource:0}: Error finding container 4bc070913a5d3b2f99f78b1195cc86f758d1f995d701a7af2f2a9a2f5f18a0c3: Status 404 returned error can't find the container with id 4bc070913a5d3b2f99f78b1195cc86f758d1f995d701a7af2f2a9a2f5f18a0c3 Sep 10 13:21:35.089118 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:35.089090 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gtqlh_52fceedd-00f2-4943-ae47-dec51216d8e4/console-operator/2.log" Sep 10 13:21:35.090979 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:35.090941 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9242g" event={"ID":"6f1d2f01-1377-4ca2-9b34-c70d033a0920","Type":"ContainerStarted","Data":"1f23f824c995092250a9c2e6f6b3c2df2166db1d1c2e00c0bab26e39e25add3b"} Sep 10 13:21:35.091111 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:35.091054 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9242g" event={"ID":"6f1d2f01-1377-4ca2-9b34-c70d033a0920","Type":"ContainerStarted","Data":"4bc070913a5d3b2f99f78b1195cc86f758d1f995d701a7af2f2a9a2f5f18a0c3"} Sep 10 13:21:36.096062 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:36.096027 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-v2hld" event={"ID":"faa8bfdc-9c79-4c12-8ec9-3cd8639d0793","Type":"ContainerStarted","Data":"c34d81790fbd63278eb630c33f52e50227ca66a8174cb87f3b13db7127bee338"} Sep 10 13:21:37.102049 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:37.102011 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9242g" event={"ID":"6f1d2f01-1377-4ca2-9b34-c70d033a0920","Type":"ContainerStarted","Data":"fb89eaafeb9d95702a34b11b3ad3d33698b650070074c3a68bc72cf1ff064bc5"} Sep 10 13:21:37.915964 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:37.915936 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:37.987989 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:37.987956 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:38.107677 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:38.107588 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9242g" event={"ID":"6f1d2f01-1377-4ca2-9b34-c70d033a0920","Type":"ContainerStarted","Data":"1aa9d3aa34fdc5768b4a9dd3ef382de8cc29d510edc779218c1a1abccb2c4b64"} Sep 10 13:21:38.130910 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:38.130849 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-v2hld" podStartSLOduration=66.896152796 podStartE2EDuration="1m11.130825194s" totalImagesPullingTime="4.234672398s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:20:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:21:36.1384689 +0000 UTC m=+206.033186315" watchObservedRunningTime="2026-09-10 13:21:38.130825194 +0000 UTC m=+208.025542612" Sep 10 13:21:38.131527 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:38.131497 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-9242g" podStartSLOduration=16.770204224 podStartE2EDuration="20.131486044s" totalImagesPullingTime="3.36128182s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:21:18 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:21:38.129274898 +0000 UTC m=+208.023992317" watchObservedRunningTime="2026-09-10 13:21:38.131486044 +0000 UTC m=+208.026203461" Sep 10 13:21:38.953773 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:38.953734 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:40.131075 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:40.131021 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-gtqlh" Sep 10 13:21:40.131505 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:40.131123 2560 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-gtqlh" Sep 10 13:21:40.131505 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:40.131455 2560 scope.go:117] "RemoveContainer" containerID="708b62fe219deb467b6409953de1babbae619f2b46f7c84e0f31bd6c97097db9" Sep 10 13:21:40.131630 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:40.131610 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-gtqlh_openshift-console-operator(52fceedd-00f2-4943-ae47-dec51216d8e4)\"" pod="openshift-console-operator/console-operator-85b454874b-gtqlh" podUID="52fceedd-00f2-4943-ae47-dec51216d8e4" Sep 10 13:21:41.746740 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:41.746690 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-metrics-certs\") pod \"router-default-5787c86768-n9xzb\" (UID: \"b19de8c5-85f7-4f5e-9011-8f38d0b80a69\") " pod="openshift-ingress/router-default-5787c86768-n9xzb" Sep 10 13:21:41.747249 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:41.746748 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7fe46372-65c2-451c-9fba-11399f8c753d-metrics-tls\") pod \"dns-default-2tqpb\" (UID: \"7fe46372-65c2-451c-9fba-11399f8c753d\") " pod="openshift-dns/dns-default-2tqpb" Sep 10 13:21:41.747249 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:41.746773 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3ef3618e-e54b-46bf-8347-69b8e72720fd-registry-tls\") pod \"image-registry-7d5587b5d-tzd9q\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" Sep 10 13:21:41.747249 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:41.746825 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e7fc744a-ed33-44a7-94f1-19bbd01b9c50-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-qw7g8\" (UID: \"e7fc744a-ed33-44a7-94f1-19bbd01b9c50\") " pod="openshift-network-console/networking-console-plugin-79977f9844-qw7g8" Sep 10 13:21:41.747249 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:41.746872 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4a051c0c-418c-439b-94e1-181c0a84e86b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-8k875\" (UID: \"4a051c0c-418c-439b-94e1-181c0a84e86b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8k875" Sep 10 13:21:41.747249 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:41.746910 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-service-ca-bundle\") pod \"router-default-5787c86768-n9xzb\" (UID: \"b19de8c5-85f7-4f5e-9011-8f38d0b80a69\") " pod="openshift-ingress/router-default-5787c86768-n9xzb" Sep 10 13:21:41.747651 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:41.747629 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-service-ca-bundle\") pod \"router-default-5787c86768-n9xzb\" (UID: \"b19de8c5-85f7-4f5e-9011-8f38d0b80a69\") " pod="openshift-ingress/router-default-5787c86768-n9xzb" Sep 10 13:21:41.749138 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:41.749116 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7fe46372-65c2-451c-9fba-11399f8c753d-metrics-tls\") pod \"dns-default-2tqpb\" (UID: \"7fe46372-65c2-451c-9fba-11399f8c753d\") " pod="openshift-dns/dns-default-2tqpb" Sep 10 13:21:41.749661 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:41.749641 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e7fc744a-ed33-44a7-94f1-19bbd01b9c50-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-qw7g8\" (UID: \"e7fc744a-ed33-44a7-94f1-19bbd01b9c50\") " pod="openshift-network-console/networking-console-plugin-79977f9844-qw7g8" Sep 10 13:21:41.749741 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:41.749660 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3ef3618e-e54b-46bf-8347-69b8e72720fd-registry-tls\") pod \"image-registry-7d5587b5d-tzd9q\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" Sep 10 13:21:41.749741 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:41.749668 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b19de8c5-85f7-4f5e-9011-8f38d0b80a69-metrics-certs\") pod \"router-default-5787c86768-n9xzb\" (UID: \"b19de8c5-85f7-4f5e-9011-8f38d0b80a69\") " pod="openshift-ingress/router-default-5787c86768-n9xzb" Sep 10 13:21:41.749852 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:41.749811 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4a051c0c-418c-439b-94e1-181c0a84e86b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-8k875\" (UID: \"4a051c0c-418c-439b-94e1-181c0a84e86b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8k875" Sep 10 13:21:41.896957 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:41.896923 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-x85fh\"" Sep 10 13:21:41.905380 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:41.905348 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8k875" Sep 10 13:21:41.925960 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:41.925936 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gf67b\"" Sep 10 13:21:41.933137 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:41.933109 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" Sep 10 13:21:41.947511 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:41.947484 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-gwdml\"" Sep 10 13:21:41.953676 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:41.953652 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5787c86768-n9xzb" Sep 10 13:21:41.972205 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:41.971944 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-qg65m\"" Sep 10 13:21:41.977880 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:41.977673 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7k2sl\"" Sep 10 13:21:41.979523 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:41.978890 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-qw7g8" Sep 10 13:21:41.984683 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:41.984610 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2tqpb" Sep 10 13:21:42.190727 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:42.190696 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8k875"] Sep 10 13:21:42.193592 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:21:42.193561 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a051c0c_418c_439b_94e1_181c0a84e86b.slice/crio-9d30ff8da76d103f9a3dfdaa8ddf3e0474af4bffe1f477250733581540f355fc WatchSource:0}: Error finding container 9d30ff8da76d103f9a3dfdaa8ddf3e0474af4bffe1f477250733581540f355fc: Status 404 returned error can't find the container with id 9d30ff8da76d103f9a3dfdaa8ddf3e0474af4bffe1f477250733581540f355fc Sep 10 13:21:42.233674 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:42.233641 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d5587b5d-tzd9q"] Sep 10 13:21:42.239843 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:21:42.239816 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ef3618e_e54b_46bf_8347_69b8e72720fd.slice/crio-e94b1c4ef3b001d39eddfbfd572b3126038e3be76732fc728a838083216d8eea WatchSource:0}: Error finding container e94b1c4ef3b001d39eddfbfd572b3126038e3be76732fc728a838083216d8eea: Status 404 returned error can't find the container with id e94b1c4ef3b001d39eddfbfd572b3126038e3be76732fc728a838083216d8eea Sep 10 13:21:42.266482 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:42.266451 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5787c86768-n9xzb"] Sep 10 13:21:42.270380 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:21:42.270355 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb19de8c5_85f7_4f5e_9011_8f38d0b80a69.slice/crio-a04ff9a9d84d3bb9ef82a14566725899bff4a286c3b5fa602716e2ada40fee27 WatchSource:0}: Error finding container a04ff9a9d84d3bb9ef82a14566725899bff4a286c3b5fa602716e2ada40fee27: Status 404 returned error can't find the container with id a04ff9a9d84d3bb9ef82a14566725899bff4a286c3b5fa602716e2ada40fee27 Sep 10 13:21:42.280968 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:42.280946 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-qw7g8"] Sep 10 13:21:42.291437 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:21:42.291393 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7fc744a_ed33_44a7_94f1_19bbd01b9c50.slice/crio-4b3fab7dfb1ec3c3996dde6ffeaacce45b479b5f6d394b7754b74f00a185470b WatchSource:0}: Error finding container 4b3fab7dfb1ec3c3996dde6ffeaacce45b479b5f6d394b7754b74f00a185470b: Status 404 returned error can't find the container with id 4b3fab7dfb1ec3c3996dde6ffeaacce45b479b5f6d394b7754b74f00a185470b Sep 10 13:21:42.295912 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:42.295885 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2tqpb"] Sep 10 13:21:42.299757 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:21:42.299733 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fe46372_65c2_451c_9fba_11399f8c753d.slice/crio-602ef6191a51adb65e69e4a5a23b59b9293a333d4d26aa9e7f70fd9c66bc64af WatchSource:0}: Error finding container 602ef6191a51adb65e69e4a5a23b59b9293a333d4d26aa9e7f70fd9c66bc64af: Status 404 returned error can't find the container with id 602ef6191a51adb65e69e4a5a23b59b9293a333d4d26aa9e7f70fd9c66bc64af Sep 10 13:21:43.128499 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:43.128461 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" event={"ID":"3ef3618e-e54b-46bf-8347-69b8e72720fd","Type":"ContainerStarted","Data":"687a70f006d79b43df662e37474810da85aa91fa5470ed8c314412abdc473063"} Sep 10 13:21:43.128973 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:43.128521 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" event={"ID":"3ef3618e-e54b-46bf-8347-69b8e72720fd","Type":"ContainerStarted","Data":"e94b1c4ef3b001d39eddfbfd572b3126038e3be76732fc728a838083216d8eea"} Sep 10 13:21:43.129232 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:43.129211 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" Sep 10 13:21:43.130511 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:43.130480 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8k875" event={"ID":"4a051c0c-418c-439b-94e1-181c0a84e86b","Type":"ContainerStarted","Data":"9d30ff8da76d103f9a3dfdaa8ddf3e0474af4bffe1f477250733581540f355fc"} Sep 10 13:21:43.132332 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:43.132282 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2tqpb" event={"ID":"7fe46372-65c2-451c-9fba-11399f8c753d","Type":"ContainerStarted","Data":"602ef6191a51adb65e69e4a5a23b59b9293a333d4d26aa9e7f70fd9c66bc64af"} Sep 10 13:21:43.134501 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:43.134430 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-qw7g8" event={"ID":"e7fc744a-ed33-44a7-94f1-19bbd01b9c50","Type":"ContainerStarted","Data":"4b3fab7dfb1ec3c3996dde6ffeaacce45b479b5f6d394b7754b74f00a185470b"} Sep 10 13:21:43.136952 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:43.136925 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5787c86768-n9xzb" event={"ID":"b19de8c5-85f7-4f5e-9011-8f38d0b80a69","Type":"ContainerStarted","Data":"d8f59976d5d1b0f54cf447c9b3ba34622d0cae1a58ab3d2f375a1df2c248f02c"} Sep 10 13:21:43.137049 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:43.136966 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5787c86768-n9xzb" event={"ID":"b19de8c5-85f7-4f5e-9011-8f38d0b80a69","Type":"ContainerStarted","Data":"a04ff9a9d84d3bb9ef82a14566725899bff4a286c3b5fa602716e2ada40fee27"} Sep 10 13:21:43.165023 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:43.161825 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" podStartSLOduration=77.161803887 podStartE2EDuration="1m17.161803887s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:20:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:21:43.157855615 +0000 UTC m=+213.052573034" watchObservedRunningTime="2026-09-10 13:21:43.161803887 +0000 UTC m=+213.056521323" Sep 10 13:21:43.199638 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:43.199446 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5787c86768-n9xzb" podStartSLOduration=77.199425717 podStartE2EDuration="1m17.199425717s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:20:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:21:43.198293713 +0000 UTC m=+213.093011142" watchObservedRunningTime="2026-09-10 13:21:43.199425717 +0000 UTC m=+213.094143138" Sep 10 13:21:43.954170 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:43.954133 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5787c86768-n9xzb" Sep 10 13:21:43.957181 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:43.957153 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5787c86768-n9xzb" Sep 10 13:21:44.141808 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:44.141740 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5787c86768-n9xzb" Sep 10 13:21:44.142993 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:44.142969 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5787c86768-n9xzb" Sep 10 13:21:44.977740 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:44.977703 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:45.104978 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:45.104940 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:45.148193 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:45.148158 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8k875" event={"ID":"4a051c0c-418c-439b-94e1-181c0a84e86b","Type":"ContainerStarted","Data":"fcaf2f58852dde7d7ed0e2c1155eeb211fd99daed40baad4ae4f63e87e352287"} Sep 10 13:21:45.150059 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:45.150033 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2tqpb" event={"ID":"7fe46372-65c2-451c-9fba-11399f8c753d","Type":"ContainerStarted","Data":"34d54fd43e539af762f5ee5dfcdf995863f75eb8714952ab51bdcd3f59d81402"} Sep 10 13:21:45.150190 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:45.150082 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2tqpb" event={"ID":"7fe46372-65c2-451c-9fba-11399f8c753d","Type":"ContainerStarted","Data":"ca3b5d0ba5249aa9d5593064594cc56a9150ab9d37ed46ad515930998529159c"} Sep 10 13:21:45.150190 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:45.150113 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2tqpb" Sep 10 13:21:45.151705 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:45.151678 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-qw7g8" event={"ID":"e7fc744a-ed33-44a7-94f1-19bbd01b9c50","Type":"ContainerStarted","Data":"7e080f2e3de58ce660dcc5309e797f402a32ac7b82e9d1ae59c8301bd2ea7a93"} Sep 10 13:21:45.185991 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:45.185939 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-8k875" podStartSLOduration=76.88236381 podStartE2EDuration="1m19.185923224s" totalImagesPullingTime="2.303559414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:20:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:21:45.184993966 +0000 UTC m=+215.079711390" watchObservedRunningTime="2026-09-10 13:21:45.185923224 +0000 UTC m=+215.080640641" Sep 10 13:21:45.220824 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:45.220760 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-qw7g8" podStartSLOduration=77.961479337 podStartE2EDuration="1m19.220738534s" totalImagesPullingTime="1.259259197s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:20:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:21:45.220602728 +0000 UTC m=+215.115320158" watchObservedRunningTime="2026-09-10 13:21:45.220738534 +0000 UTC m=+215.115455954" Sep 10 13:21:45.257079 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:45.256982 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:45.431832 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:45.431801 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:45.966846 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:45.966806 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:46.119100 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:46.119000 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:46.278361 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:46.278326 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:46.521277 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:46.521245 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:46.652782 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:46.652747 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:50.393919 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:50.393876 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:50.590445 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:50.590409 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:50.767821 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:50.767775 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:50.916677 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:50.916642 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:51.557598 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:51.557563 2560 scope.go:117] "RemoveContainer" containerID="708b62fe219deb467b6409953de1babbae619f2b46f7c84e0f31bd6c97097db9" Sep 10 13:21:51.558009 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:21:51.557726 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-gtqlh_openshift-console-operator(52fceedd-00f2-4943-ae47-dec51216d8e4)\"" pod="openshift-console-operator/console-operator-85b454874b-gtqlh" podUID="52fceedd-00f2-4943-ae47-dec51216d8e4" Sep 10 13:21:53.602096 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:53.602061 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:55.157774 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:55.157742 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2tqpb" Sep 10 13:21:55.194842 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:55.194793 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2tqpb" podStartSLOduration=43.99530958 podStartE2EDuration="46.194778682s" totalImagesPullingTime="2.199469102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:21:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:21:45.247793141 +0000 UTC m=+215.142510560" watchObservedRunningTime="2026-09-10 13:21:55.194778682 +0000 UTC m=+225.089496099" Sep 10 13:21:55.610826 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:55.610793 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:55.815517 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:55.815484 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:56.172966 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.172930 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:56.203024 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.202997 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-zscnt"] Sep 10 13:21:56.207334 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.207316 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zscnt" Sep 10 13:21:56.215231 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.215210 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 10 13:21:56.217341 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.217310 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 10 13:21:56.217598 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.217580 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 10 13:21:56.217717 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.217691 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vt7d7\"" Sep 10 13:21:56.217833 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.217814 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 10 13:21:56.363978 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.363945 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e6173c35-bb5d-438a-be83-6da1061b6d32-node-exporter-wtmp\") pod \"node-exporter-zscnt\" (UID: \"e6173c35-bb5d-438a-be83-6da1061b6d32\") " pod="openshift-monitoring/node-exporter-zscnt" Sep 10 13:21:56.363978 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.363978 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6173c35-bb5d-438a-be83-6da1061b6d32-metrics-client-ca\") pod \"node-exporter-zscnt\" (UID: \"e6173c35-bb5d-438a-be83-6da1061b6d32\") " pod="openshift-monitoring/node-exporter-zscnt" Sep 10 13:21:56.364225 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.363999 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e6173c35-bb5d-438a-be83-6da1061b6d32-node-exporter-accelerators-collector-config\") pod \"node-exporter-zscnt\" (UID: \"e6173c35-bb5d-438a-be83-6da1061b6d32\") " pod="openshift-monitoring/node-exporter-zscnt" Sep 10 13:21:56.364225 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.364057 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e6173c35-bb5d-438a-be83-6da1061b6d32-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zscnt\" (UID: \"e6173c35-bb5d-438a-be83-6da1061b6d32\") " pod="openshift-monitoring/node-exporter-zscnt" Sep 10 13:21:56.364225 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.364100 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e6173c35-bb5d-438a-be83-6da1061b6d32-root\") pod \"node-exporter-zscnt\" (UID: \"e6173c35-bb5d-438a-be83-6da1061b6d32\") " pod="openshift-monitoring/node-exporter-zscnt" Sep 10 13:21:56.364225 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.364148 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e6173c35-bb5d-438a-be83-6da1061b6d32-node-exporter-textfile\") pod \"node-exporter-zscnt\" (UID: \"e6173c35-bb5d-438a-be83-6da1061b6d32\") " pod="openshift-monitoring/node-exporter-zscnt" Sep 10 13:21:56.364450 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.364223 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqm6t\" (UniqueName: \"kubernetes.io/projected/e6173c35-bb5d-438a-be83-6da1061b6d32-kube-api-access-pqm6t\") pod \"node-exporter-zscnt\" (UID: \"e6173c35-bb5d-438a-be83-6da1061b6d32\") " pod="openshift-monitoring/node-exporter-zscnt" Sep 10 13:21:56.364450 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.364263 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e6173c35-bb5d-438a-be83-6da1061b6d32-node-exporter-tls\") pod \"node-exporter-zscnt\" (UID: \"e6173c35-bb5d-438a-be83-6da1061b6d32\") " pod="openshift-monitoring/node-exporter-zscnt" Sep 10 13:21:56.364450 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.364282 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e6173c35-bb5d-438a-be83-6da1061b6d32-sys\") pod \"node-exporter-zscnt\" (UID: \"e6173c35-bb5d-438a-be83-6da1061b6d32\") " pod="openshift-monitoring/node-exporter-zscnt" Sep 10 13:21:56.412037 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.412005 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:56.465062 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.464991 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqm6t\" (UniqueName: \"kubernetes.io/projected/e6173c35-bb5d-438a-be83-6da1061b6d32-kube-api-access-pqm6t\") pod \"node-exporter-zscnt\" (UID: \"e6173c35-bb5d-438a-be83-6da1061b6d32\") " pod="openshift-monitoring/node-exporter-zscnt" Sep 10 13:21:56.465062 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.465033 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e6173c35-bb5d-438a-be83-6da1061b6d32-node-exporter-tls\") pod \"node-exporter-zscnt\" (UID: \"e6173c35-bb5d-438a-be83-6da1061b6d32\") " pod="openshift-monitoring/node-exporter-zscnt" Sep 10 13:21:56.465062 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.465047 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e6173c35-bb5d-438a-be83-6da1061b6d32-sys\") pod \"node-exporter-zscnt\" (UID: \"e6173c35-bb5d-438a-be83-6da1061b6d32\") " pod="openshift-monitoring/node-exporter-zscnt" Sep 10 13:21:56.465062 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.465067 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e6173c35-bb5d-438a-be83-6da1061b6d32-node-exporter-wtmp\") pod \"node-exporter-zscnt\" (UID: \"e6173c35-bb5d-438a-be83-6da1061b6d32\") " pod="openshift-monitoring/node-exporter-zscnt" Sep 10 13:21:56.465357 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.465089 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6173c35-bb5d-438a-be83-6da1061b6d32-metrics-client-ca\") pod \"node-exporter-zscnt\" (UID: \"e6173c35-bb5d-438a-be83-6da1061b6d32\") " pod="openshift-monitoring/node-exporter-zscnt" Sep 10 13:21:56.465357 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.465116 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e6173c35-bb5d-438a-be83-6da1061b6d32-node-exporter-accelerators-collector-config\") pod \"node-exporter-zscnt\" (UID: \"e6173c35-bb5d-438a-be83-6da1061b6d32\") " pod="openshift-monitoring/node-exporter-zscnt" Sep 10 13:21:56.465357 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.465133 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e6173c35-bb5d-438a-be83-6da1061b6d32-sys\") pod \"node-exporter-zscnt\" (UID: \"e6173c35-bb5d-438a-be83-6da1061b6d32\") " pod="openshift-monitoring/node-exporter-zscnt" Sep 10 13:21:56.465357 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.465155 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e6173c35-bb5d-438a-be83-6da1061b6d32-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zscnt\" (UID: \"e6173c35-bb5d-438a-be83-6da1061b6d32\") " pod="openshift-monitoring/node-exporter-zscnt" Sep 10 13:21:56.465357 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.465198 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e6173c35-bb5d-438a-be83-6da1061b6d32-root\") pod \"node-exporter-zscnt\" (UID: \"e6173c35-bb5d-438a-be83-6da1061b6d32\") " pod="openshift-monitoring/node-exporter-zscnt" Sep 10 13:21:56.465357 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.465223 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e6173c35-bb5d-438a-be83-6da1061b6d32-node-exporter-wtmp\") pod \"node-exporter-zscnt\" (UID: \"e6173c35-bb5d-438a-be83-6da1061b6d32\") " pod="openshift-monitoring/node-exporter-zscnt" Sep 10 13:21:56.465357 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.465249 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e6173c35-bb5d-438a-be83-6da1061b6d32-node-exporter-textfile\") pod \"node-exporter-zscnt\" (UID: \"e6173c35-bb5d-438a-be83-6da1061b6d32\") " pod="openshift-monitoring/node-exporter-zscnt" Sep 10 13:21:56.465357 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.465270 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e6173c35-bb5d-438a-be83-6da1061b6d32-root\") pod \"node-exporter-zscnt\" (UID: \"e6173c35-bb5d-438a-be83-6da1061b6d32\") " pod="openshift-monitoring/node-exporter-zscnt" Sep 10 13:21:56.465805 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.465651 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e6173c35-bb5d-438a-be83-6da1061b6d32-node-exporter-textfile\") pod \"node-exporter-zscnt\" (UID: \"e6173c35-bb5d-438a-be83-6da1061b6d32\") " pod="openshift-monitoring/node-exporter-zscnt" Sep 10 13:21:56.465805 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.465760 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e6173c35-bb5d-438a-be83-6da1061b6d32-node-exporter-accelerators-collector-config\") pod \"node-exporter-zscnt\" (UID: \"e6173c35-bb5d-438a-be83-6da1061b6d32\") " pod="openshift-monitoring/node-exporter-zscnt" Sep 10 13:21:56.465903 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.465843 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6173c35-bb5d-438a-be83-6da1061b6d32-metrics-client-ca\") pod \"node-exporter-zscnt\" (UID: \"e6173c35-bb5d-438a-be83-6da1061b6d32\") " pod="openshift-monitoring/node-exporter-zscnt" Sep 10 13:21:56.467468 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.467449 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e6173c35-bb5d-438a-be83-6da1061b6d32-node-exporter-tls\") pod \"node-exporter-zscnt\" (UID: \"e6173c35-bb5d-438a-be83-6da1061b6d32\") " pod="openshift-monitoring/node-exporter-zscnt" Sep 10 13:21:56.467556 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.467489 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e6173c35-bb5d-438a-be83-6da1061b6d32-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zscnt\" (UID: \"e6173c35-bb5d-438a-be83-6da1061b6d32\") " pod="openshift-monitoring/node-exporter-zscnt" Sep 10 13:21:56.475824 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.475796 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqm6t\" (UniqueName: \"kubernetes.io/projected/e6173c35-bb5d-438a-be83-6da1061b6d32-kube-api-access-pqm6t\") pod \"node-exporter-zscnt\" (UID: \"e6173c35-bb5d-438a-be83-6da1061b6d32\") " pod="openshift-monitoring/node-exporter-zscnt" Sep 10 13:21:56.515963 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.515926 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zscnt" Sep 10 13:21:56.525558 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:21:56.525531 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6173c35_bb5d_438a_be83_6da1061b6d32.slice/crio-d8f2260d19ebe18e4cfb6b1335251044bb51339af66deac87007ba43e756783e WatchSource:0}: Error finding container d8f2260d19ebe18e4cfb6b1335251044bb51339af66deac87007ba43e756783e: Status 404 returned error can't find the container with id d8f2260d19ebe18e4cfb6b1335251044bb51339af66deac87007ba43e756783e Sep 10 13:21:56.561057 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.561002 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:56.704949 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.704917 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:56.959847 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:56.959818 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:57.135362 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.135328 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:57.200483 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.200446 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zscnt" event={"ID":"e6173c35-bb5d-438a-be83-6da1061b6d32","Type":"ContainerStarted","Data":"d8f2260d19ebe18e4cfb6b1335251044bb51339af66deac87007ba43e756783e"} Sep 10 13:21:57.281377 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.281290 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:57.335767 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.335685 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 13:21:57.339279 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.339240 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.348432 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.345769 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 10 13:21:57.348432 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.345919 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 10 13:21:57.348432 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.345946 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 10 13:21:57.348432 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.347265 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 10 13:21:57.348432 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.347296 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 10 13:21:57.348821 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.348472 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 10 13:21:57.348821 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.348645 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-5dpvg\"" Sep 10 13:21:57.348821 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.348691 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 10 13:21:57.348972 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.348828 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 10 13:21:57.357751 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.357505 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 10 13:21:57.382592 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.382378 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 13:21:57.474681 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.474651 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/21369021-706c-4fdc-9d1b-404a0994b1f5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.474857 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.474697 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.474857 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.474726 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21369021-706c-4fdc-9d1b-404a0994b1f5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.474857 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.474749 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21369021-706c-4fdc-9d1b-404a0994b1f5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.474857 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.474776 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4xzf\" (UniqueName: \"kubernetes.io/projected/21369021-706c-4fdc-9d1b-404a0994b1f5-kube-api-access-d4xzf\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.474857 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.474806 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.474857 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.474832 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.475192 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.474898 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/21369021-706c-4fdc-9d1b-404a0994b1f5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.475192 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.474933 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-config-volume\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.475192 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.474955 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.475192 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.475001 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-web-config\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.475192 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.475028 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/21369021-706c-4fdc-9d1b-404a0994b1f5-config-out\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.475192 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.475058 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.575935 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.575852 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/21369021-706c-4fdc-9d1b-404a0994b1f5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.575935 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.575903 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.576170 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.575962 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21369021-706c-4fdc-9d1b-404a0994b1f5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.576170 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.576000 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21369021-706c-4fdc-9d1b-404a0994b1f5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.576170 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.576030 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4xzf\" (UniqueName: \"kubernetes.io/projected/21369021-706c-4fdc-9d1b-404a0994b1f5-kube-api-access-d4xzf\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.576170 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.576064 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.576170 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.576090 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.576170 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.576162 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/21369021-706c-4fdc-9d1b-404a0994b1f5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.576508 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.576194 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-config-volume\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.576508 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.576214 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.576508 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.576259 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-web-config\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.576508 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.576283 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/21369021-706c-4fdc-9d1b-404a0994b1f5-config-out\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.576508 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.576306 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.578646 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.577132 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21369021-706c-4fdc-9d1b-404a0994b1f5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.578646 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.578282 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21369021-706c-4fdc-9d1b-404a0994b1f5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.578925 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.578901 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/21369021-706c-4fdc-9d1b-404a0994b1f5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.580011 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.579970 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.581039 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.580535 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.581039 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.580807 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.581039 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.580880 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.581039 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.580921 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-web-config\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.581039 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.580937 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-config-volume\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.581646 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.581623 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.581897 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.581877 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/21369021-706c-4fdc-9d1b-404a0994b1f5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.581974 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.581924 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/21369021-706c-4fdc-9d1b-404a0994b1f5-config-out\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.593717 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.593695 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4xzf\" (UniqueName: \"kubernetes.io/projected/21369021-706c-4fdc-9d1b-404a0994b1f5-kube-api-access-d4xzf\") pod \"alertmanager-main-0\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.660667 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.660632 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:21:57.742468 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:57.741258 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:58.119753 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:58.119721 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 13:21:58.120172 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:21:58.120151 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21369021_706c_4fdc_9d1b_404a0994b1f5.slice/crio-da7478386c11ea36ff719ad35203571b3d8bcbb0d9f36f19c9088023e3c2c96d WatchSource:0}: Error finding container da7478386c11ea36ff719ad35203571b3d8bcbb0d9f36f19c9088023e3c2c96d: Status 404 returned error can't find the container with id da7478386c11ea36ff719ad35203571b3d8bcbb0d9f36f19c9088023e3c2c96d Sep 10 13:21:58.206228 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:58.206188 2560 generic.go:356] "Generic (PLEG): container finished" podID="e6173c35-bb5d-438a-be83-6da1061b6d32" containerID="b1d5b0354e8c37581a64658cf0d960acc581eb6c197cad1d5df4ed6fac42fe1d" exitCode=0 Sep 10 13:21:58.206228 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:58.206223 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zscnt" event={"ID":"e6173c35-bb5d-438a-be83-6da1061b6d32","Type":"ContainerDied","Data":"b1d5b0354e8c37581a64658cf0d960acc581eb6c197cad1d5df4ed6fac42fe1d"} Sep 10 13:21:58.207652 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:58.207630 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21369021-706c-4fdc-9d1b-404a0994b1f5","Type":"ContainerStarted","Data":"da7478386c11ea36ff719ad35203571b3d8bcbb0d9f36f19c9088023e3c2c96d"} Sep 10 13:21:58.501295 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:58.501262 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:58.671559 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:58.671519 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:58.869425 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:58.869314 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:59.217223 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:59.217174 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zscnt" event={"ID":"e6173c35-bb5d-438a-be83-6da1061b6d32","Type":"ContainerStarted","Data":"e8be0a50402d244b09a31fded41abd1f270b84594ab6ba37266f52d29cd43b93"} Sep 10 13:21:59.217717 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:59.217234 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zscnt" event={"ID":"e6173c35-bb5d-438a-be83-6da1061b6d32","Type":"ContainerStarted","Data":"5c497b3a98f171f32fbf2e80efab31a9c078a7794383419b807f5e358e089dc3"} Sep 10 13:21:59.289824 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:59.289762 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-zscnt" podStartSLOduration=2.069589199 podStartE2EDuration="3.289740401s" totalImagesPullingTime="1.157301648s" totalInitContainerRuntime="62.849554ms" isStatefulPod=true podCreationTimestamp="2026-09-10 13:21:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:21:59.286897321 +0000 UTC m=+229.181614740" watchObservedRunningTime="2026-09-10 13:21:59.289740401 +0000 UTC m=+229.184457823" Sep 10 13:21:59.350270 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:59.350232 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:59.582749 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:59.582650 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:21:59.703421 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:21:59.703369 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:00.064863 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:00.064826 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:00.222060 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:00.222027 2560 generic.go:356] "Generic (PLEG): container finished" podID="21369021-706c-4fdc-9d1b-404a0994b1f5" containerID="44a8848183abe6c9a26263201814dde6874c10b0d2057d7cadd85691551c1853" exitCode=0 Sep 10 13:22:00.222554 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:00.222121 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21369021-706c-4fdc-9d1b-404a0994b1f5","Type":"ContainerDied","Data":"44a8848183abe6c9a26263201814dde6874c10b0d2057d7cadd85691551c1853"} Sep 10 13:22:00.710023 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:00.709986 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:00.944137 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:00.944069 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:01.938235 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:01.938193 2560 patch_prober.go:28] interesting pod/image-registry-7d5587b5d-tzd9q container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 10 13:22:01.938635 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:01.938246 2560 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" podUID="3ef3618e-e54b-46bf-8347-69b8e72720fd" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 13:22:02.864219 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:02.864179 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:03.023648 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:03.023544 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:03.079479 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:03.079449 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-g64cr" Sep 10 13:22:03.236453 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:03.236422 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21369021-706c-4fdc-9d1b-404a0994b1f5","Type":"ContainerStarted","Data":"8c7ccae57ac9c27e6a141e6074a3dd21a85da6865eb6b48f75162380be4dfa3f"} Sep 10 13:22:03.236629 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:03.236482 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21369021-706c-4fdc-9d1b-404a0994b1f5","Type":"ContainerStarted","Data":"c51dccbb25ffcc1549bf9268cf7b5f920d0b4f5e016774a370e3434534837134"} Sep 10 13:22:03.236629 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:03.236510 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21369021-706c-4fdc-9d1b-404a0994b1f5","Type":"ContainerStarted","Data":"86114bbcd75b5c080391e5c8fd632f2d1fa0e37cab2a730bb52e32526c04d565"} Sep 10 13:22:03.236629 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:03.236532 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21369021-706c-4fdc-9d1b-404a0994b1f5","Type":"ContainerStarted","Data":"b298bbd91d08880a70085b578023f9d2e3de295e05170e60136b15fcb742abc7"} Sep 10 13:22:03.236629 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:03.236554 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21369021-706c-4fdc-9d1b-404a0994b1f5","Type":"ContainerStarted","Data":"b3e37610262018c102dbb174b1be06d334ac55e64e036c5c7733facdf1927479"} Sep 10 13:22:04.243518 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:04.243474 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21369021-706c-4fdc-9d1b-404a0994b1f5","Type":"ContainerStarted","Data":"b9a091d909447e385c472c60b76ce00e13b3d49d70299839eccbe194831f7349"} Sep 10 13:22:04.294828 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:04.294771 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.715357303 podStartE2EDuration="7.294755266s" totalImagesPullingTime="4.561424855s" totalInitContainerRuntime="17.973108ms" isStatefulPod=false podCreationTimestamp="2026-09-10 13:21:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:22:04.291684677 +0000 UTC m=+234.186402094" watchObservedRunningTime="2026-09-10 13:22:04.294755266 +0000 UTC m=+234.189472684" Sep 10 13:22:04.494896 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:04.494796 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:04.653996 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:04.653964 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:05.157166 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:05.157131 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" Sep 10 13:22:05.557599 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:05.557566 2560 scope.go:117] "RemoveContainer" containerID="708b62fe219deb467b6409953de1babbae619f2b46f7c84e0f31bd6c97097db9" Sep 10 13:22:05.610548 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:05.610525 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:05.921635 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:05.921598 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:06.253892 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:06.253822 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gtqlh_52fceedd-00f2-4943-ae47-dec51216d8e4/console-operator/2.log" Sep 10 13:22:06.254127 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:06.253907 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-gtqlh" event={"ID":"52fceedd-00f2-4943-ae47-dec51216d8e4","Type":"ContainerStarted","Data":"0adfa60ed793b31919aee5393ea266b5e056c112f26bd4f01f8771dc9f34f12f"} Sep 10 13:22:06.254200 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:06.254181 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-gtqlh" Sep 10 13:22:06.259044 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:06.259014 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-gtqlh" Sep 10 13:22:06.311742 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:06.311683 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-gtqlh" podStartSLOduration=94.346389063 podStartE2EDuration="1m40.311663114s" totalImagesPullingTime="5.965274051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:20:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:22:06.309467895 +0000 UTC m=+236.204185312" watchObservedRunningTime="2026-09-10 13:22:06.311663114 +0000 UTC m=+236.206380524" Sep 10 13:22:06.469680 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:06.469646 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:06.638562 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:06.638473 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:06.846890 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:06.846855 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:06.959695 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:06.959660 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:07.343429 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:07.343312 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:07.740953 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:07.740920 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:07.960646 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:07.960613 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:08.135436 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:08.135331 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:08.457503 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:08.457470 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:08.932046 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:08.932008 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:09.076534 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:09.076500 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:09.122248 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:09.122212 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7d5587b5d-tzd9q"] Sep 10 13:22:09.308788 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:09.308700 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:09.476448 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:09.476380 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:11.401651 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:11.401605 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:11.689431 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:11.689356 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:13.481067 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:13.481029 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:13.595175 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:13.595142 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:14.492506 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:14.492470 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:14.640708 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:14.640678 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:15.701395 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:15.701357 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:15.951793 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:15.951711 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:20.638018 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:20.637969 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:20.921349 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:20.921319 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:21.752211 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:21.752178 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:22.067010 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:22.066913 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:23.137707 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:23.137670 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:23.407894 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:23.407815 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:24.565045 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:24.565017 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:24.923984 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:24.923952 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:25.104144 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:25.104112 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:25.795767 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:25.795735 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:26.162678 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:26.162581 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:28.340946 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:28.340909 2560 generic.go:356] "Generic (PLEG): container finished" podID="9c97ef99-3ac3-4334-aba7-f9de4a6cccf6" containerID="bb3dc221e8210715ee059bf26edbacb7757acd0bd9ccd447fa617cac0c271b57" exitCode=0 Sep 10 13:22:28.341314 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:28.340967 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4hmcg" event={"ID":"9c97ef99-3ac3-4334-aba7-f9de4a6cccf6","Type":"ContainerDied","Data":"bb3dc221e8210715ee059bf26edbacb7757acd0bd9ccd447fa617cac0c271b57"} Sep 10 13:22:28.341314 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:28.341270 2560 scope.go:117] "RemoveContainer" containerID="bb3dc221e8210715ee059bf26edbacb7757acd0bd9ccd447fa617cac0c271b57" Sep 10 13:22:29.346864 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:29.346830 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-4hmcg" event={"ID":"9c97ef99-3ac3-4334-aba7-f9de4a6cccf6","Type":"ContainerStarted","Data":"9dfb09dd6cfdd0602182d92c329a54c1d4aaeea19551f6f6c913f3d54ea9f928"} Sep 10 13:22:31.722036 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:31.721996 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:31.926802 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:31.926769 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:33.381077 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:33.381000 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:33.560798 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:33.560749 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:34.141922 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:34.141879 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" podUID="3ef3618e-e54b-46bf-8347-69b8e72720fd" containerName="registry" containerID="cri-o://687a70f006d79b43df662e37474810da85aa91fa5470ed8c314412abdc473063" gracePeriod=30 Sep 10 13:22:34.368259 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:34.368225 2560 generic.go:356] "Generic (PLEG): container finished" podID="3ef3618e-e54b-46bf-8347-69b8e72720fd" containerID="687a70f006d79b43df662e37474810da85aa91fa5470ed8c314412abdc473063" exitCode=0 Sep 10 13:22:34.368446 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:34.368274 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" event={"ID":"3ef3618e-e54b-46bf-8347-69b8e72720fd","Type":"ContainerDied","Data":"687a70f006d79b43df662e37474810da85aa91fa5470ed8c314412abdc473063"} Sep 10 13:22:34.425425 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:34.425376 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" Sep 10 13:22:34.484708 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:34.484671 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3ef3618e-e54b-46bf-8347-69b8e72720fd-ca-trust-extracted\") pod \"3ef3618e-e54b-46bf-8347-69b8e72720fd\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " Sep 10 13:22:34.484708 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:34.484709 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3ef3618e-e54b-46bf-8347-69b8e72720fd-bound-sa-token\") pod \"3ef3618e-e54b-46bf-8347-69b8e72720fd\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " Sep 10 13:22:34.484960 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:34.484725 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3ef3618e-e54b-46bf-8347-69b8e72720fd-registry-certificates\") pod \"3ef3618e-e54b-46bf-8347-69b8e72720fd\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " Sep 10 13:22:34.484960 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:34.484754 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tbcrn\" (UniqueName: \"kubernetes.io/projected/3ef3618e-e54b-46bf-8347-69b8e72720fd-kube-api-access-tbcrn\") pod \"3ef3618e-e54b-46bf-8347-69b8e72720fd\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " Sep 10 13:22:34.484960 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:34.484777 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3ef3618e-e54b-46bf-8347-69b8e72720fd-installation-pull-secrets\") pod \"3ef3618e-e54b-46bf-8347-69b8e72720fd\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " Sep 10 13:22:34.484960 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:34.484795 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3ef3618e-e54b-46bf-8347-69b8e72720fd-registry-tls\") pod \"3ef3618e-e54b-46bf-8347-69b8e72720fd\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " Sep 10 13:22:34.484960 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:34.484815 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3ef3618e-e54b-46bf-8347-69b8e72720fd-trusted-ca\") pod \"3ef3618e-e54b-46bf-8347-69b8e72720fd\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " Sep 10 13:22:34.484960 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:34.484858 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3ef3618e-e54b-46bf-8347-69b8e72720fd-image-registry-private-configuration\") pod \"3ef3618e-e54b-46bf-8347-69b8e72720fd\" (UID: \"3ef3618e-e54b-46bf-8347-69b8e72720fd\") " Sep 10 13:22:34.485323 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:34.485298 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ef3618e-e54b-46bf-8347-69b8e72720fd-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "3ef3618e-e54b-46bf-8347-69b8e72720fd" (UID: "3ef3618e-e54b-46bf-8347-69b8e72720fd"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:22:34.485457 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:34.485427 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ef3618e-e54b-46bf-8347-69b8e72720fd-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "3ef3618e-e54b-46bf-8347-69b8e72720fd" (UID: "3ef3618e-e54b-46bf-8347-69b8e72720fd"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:22:34.487642 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:34.487580 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ef3618e-e54b-46bf-8347-69b8e72720fd-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "3ef3618e-e54b-46bf-8347-69b8e72720fd" (UID: "3ef3618e-e54b-46bf-8347-69b8e72720fd"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:22:34.487772 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:34.487726 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ef3618e-e54b-46bf-8347-69b8e72720fd-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "3ef3618e-e54b-46bf-8347-69b8e72720fd" (UID: "3ef3618e-e54b-46bf-8347-69b8e72720fd"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:22:34.487842 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:34.487775 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ef3618e-e54b-46bf-8347-69b8e72720fd-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "3ef3618e-e54b-46bf-8347-69b8e72720fd" (UID: "3ef3618e-e54b-46bf-8347-69b8e72720fd"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 13:22:34.487842 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:34.487807 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ef3618e-e54b-46bf-8347-69b8e72720fd-kube-api-access-tbcrn" (OuterVolumeSpecName: "kube-api-access-tbcrn") pod "3ef3618e-e54b-46bf-8347-69b8e72720fd" (UID: "3ef3618e-e54b-46bf-8347-69b8e72720fd"). InnerVolumeSpecName "kube-api-access-tbcrn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:22:34.488304 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:34.488273 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ef3618e-e54b-46bf-8347-69b8e72720fd-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "3ef3618e-e54b-46bf-8347-69b8e72720fd" (UID: "3ef3618e-e54b-46bf-8347-69b8e72720fd"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 13:22:34.493916 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:34.493891 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ef3618e-e54b-46bf-8347-69b8e72720fd-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "3ef3618e-e54b-46bf-8347-69b8e72720fd" (UID: "3ef3618e-e54b-46bf-8347-69b8e72720fd"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:22:34.585733 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:34.585695 2560 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3ef3618e-e54b-46bf-8347-69b8e72720fd-ca-trust-extracted\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:22:34.585733 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:34.585726 2560 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3ef3618e-e54b-46bf-8347-69b8e72720fd-bound-sa-token\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:22:34.585733 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:34.585739 2560 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3ef3618e-e54b-46bf-8347-69b8e72720fd-registry-certificates\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:22:34.585969 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:34.585751 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbcrn\" (UniqueName: \"kubernetes.io/projected/3ef3618e-e54b-46bf-8347-69b8e72720fd-kube-api-access-tbcrn\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:22:34.585969 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:34.585763 2560 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3ef3618e-e54b-46bf-8347-69b8e72720fd-installation-pull-secrets\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:22:34.585969 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:34.585774 2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3ef3618e-e54b-46bf-8347-69b8e72720fd-registry-tls\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:22:34.585969 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:34.585785 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3ef3618e-e54b-46bf-8347-69b8e72720fd-trusted-ca\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:22:34.585969 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:34.585796 2560 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3ef3618e-e54b-46bf-8347-69b8e72720fd-image-registry-private-configuration\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:22:35.373820 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:35.373791 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" Sep 10 13:22:35.374018 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:35.373790 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d5587b5d-tzd9q" event={"ID":"3ef3618e-e54b-46bf-8347-69b8e72720fd","Type":"ContainerDied","Data":"e94b1c4ef3b001d39eddfbfd572b3126038e3be76732fc728a838083216d8eea"} Sep 10 13:22:35.374018 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:35.373921 2560 scope.go:117] "RemoveContainer" containerID="687a70f006d79b43df662e37474810da85aa91fa5470ed8c314412abdc473063" Sep 10 13:22:35.425655 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:35.425625 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7d5587b5d-tzd9q"] Sep 10 13:22:35.432167 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:35.432144 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7d5587b5d-tzd9q"] Sep 10 13:22:36.020895 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:36.020862 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:36.443505 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:36.443447 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:36.561728 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:36.561697 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ef3618e-e54b-46bf-8347-69b8e72720fd" path="/var/lib/kubelet/pods/3ef3618e-e54b-46bf-8347-69b8e72720fd/volumes" Sep 10 13:22:37.122835 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:37.122799 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:37.270702 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:37.270662 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:37.969255 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:37.969218 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:38.119439 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:38.119381 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:38.670049 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:38.670013 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:38.856872 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:38.856832 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:22:43.405171 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:43.405138 2560 generic.go:356] "Generic (PLEG): container finished" podID="4fe9733a-2777-4507-af4b-29f330350747" containerID="50865a8e04de1851b987b68f47802430a3bb87d07940542a56ac04af290a751c" exitCode=0 Sep 10 13:22:43.405573 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:43.405210 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-mnksz" event={"ID":"4fe9733a-2777-4507-af4b-29f330350747","Type":"ContainerDied","Data":"50865a8e04de1851b987b68f47802430a3bb87d07940542a56ac04af290a751c"} Sep 10 13:22:43.405617 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:43.405582 2560 scope.go:117] "RemoveContainer" containerID="50865a8e04de1851b987b68f47802430a3bb87d07940542a56ac04af290a751c" Sep 10 13:22:44.410212 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:44.410179 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-mnksz" event={"ID":"4fe9733a-2777-4507-af4b-29f330350747","Type":"ContainerStarted","Data":"a3dee7fef532ad84556d88485718ba40ceb92e8b68f4e96cde652d65847d7fb5"} Sep 10 13:22:47.426143 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:47.426103 2560 generic.go:356] "Generic (PLEG): container finished" podID="d763e40a-94ec-4167-8962-c355da8574d0" containerID="b1c768f0e64cbdcbc26be0e8add260c967b2a033527dc195cc81ade87e2139ee" exitCode=0 Sep 10 13:22:47.426143 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:47.426129 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bdnkb" event={"ID":"d763e40a-94ec-4167-8962-c355da8574d0","Type":"ContainerDied","Data":"b1c768f0e64cbdcbc26be0e8add260c967b2a033527dc195cc81ade87e2139ee"} Sep 10 13:22:47.426719 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:47.426510 2560 scope.go:117] "RemoveContainer" containerID="b1c768f0e64cbdcbc26be0e8add260c967b2a033527dc195cc81ade87e2139ee" Sep 10 13:22:48.432085 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:22:48.432043 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-bdnkb" event={"ID":"d763e40a-94ec-4167-8962-c355da8574d0","Type":"ContainerStarted","Data":"33b6efd7680758e2163f528f46fac2327e229297008833cac5da9ba2a2ebee37"} Sep 10 13:23:10.433154 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:10.433127 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gtqlh_52fceedd-00f2-4943-ae47-dec51216d8e4/console-operator/2.log" Sep 10 13:23:10.434505 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:10.434479 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gtqlh_52fceedd-00f2-4943-ae47-dec51216d8e4/console-operator/2.log" Sep 10 13:23:10.447689 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:10.447670 2560 kubelet.go:1648] "Image garbage collection succeeded" Sep 10 13:23:15.802126 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:15.802085 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:23:15.878830 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:15.878789 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:23:16.771557 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:16.771522 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 13:23:16.771861 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:16.771807 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="21369021-706c-4fdc-9d1b-404a0994b1f5" containerName="alertmanager" containerID="cri-o://b3e37610262018c102dbb174b1be06d334ac55e64e036c5c7733facdf1927479" gracePeriod=120 Sep 10 13:23:16.772024 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:16.771904 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="21369021-706c-4fdc-9d1b-404a0994b1f5" containerName="prom-label-proxy" containerID="cri-o://b9a091d909447e385c472c60b76ce00e13b3d49d70299839eccbe194831f7349" gracePeriod=120 Sep 10 13:23:16.772024 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:16.771924 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="21369021-706c-4fdc-9d1b-404a0994b1f5" containerName="kube-rbac-proxy" containerID="cri-o://c51dccbb25ffcc1549bf9268cf7b5f920d0b4f5e016774a370e3434534837134" gracePeriod=120 Sep 10 13:23:16.772024 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:16.771953 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="21369021-706c-4fdc-9d1b-404a0994b1f5" containerName="config-reloader" containerID="cri-o://b298bbd91d08880a70085b578023f9d2e3de295e05170e60136b15fcb742abc7" gracePeriod=120 Sep 10 13:23:16.772232 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:16.771924 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="21369021-706c-4fdc-9d1b-404a0994b1f5" containerName="kube-rbac-proxy-web" containerID="cri-o://86114bbcd75b5c080391e5c8fd632f2d1fa0e37cab2a730bb52e32526c04d565" gracePeriod=120 Sep 10 13:23:16.772232 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:16.772079 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="21369021-706c-4fdc-9d1b-404a0994b1f5" containerName="kube-rbac-proxy-metric" containerID="cri-o://8c7ccae57ac9c27e6a141e6074a3dd21a85da6865eb6b48f75162380be4dfa3f" gracePeriod=120 Sep 10 13:23:17.550986 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:17.550956 2560 generic.go:356] "Generic (PLEG): container finished" podID="21369021-706c-4fdc-9d1b-404a0994b1f5" containerID="b9a091d909447e385c472c60b76ce00e13b3d49d70299839eccbe194831f7349" exitCode=0 Sep 10 13:23:17.550986 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:17.550981 2560 generic.go:356] "Generic (PLEG): container finished" podID="21369021-706c-4fdc-9d1b-404a0994b1f5" containerID="8c7ccae57ac9c27e6a141e6074a3dd21a85da6865eb6b48f75162380be4dfa3f" exitCode=0 Sep 10 13:23:17.550986 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:17.550987 2560 generic.go:356] "Generic (PLEG): container finished" podID="21369021-706c-4fdc-9d1b-404a0994b1f5" containerID="c51dccbb25ffcc1549bf9268cf7b5f920d0b4f5e016774a370e3434534837134" exitCode=0 Sep 10 13:23:17.550986 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:17.550993 2560 generic.go:356] "Generic (PLEG): container finished" podID="21369021-706c-4fdc-9d1b-404a0994b1f5" containerID="b298bbd91d08880a70085b578023f9d2e3de295e05170e60136b15fcb742abc7" exitCode=0 Sep 10 13:23:17.551476 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:17.550999 2560 generic.go:356] "Generic (PLEG): container finished" podID="21369021-706c-4fdc-9d1b-404a0994b1f5" containerID="b3e37610262018c102dbb174b1be06d334ac55e64e036c5c7733facdf1927479" exitCode=0 Sep 10 13:23:17.551476 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:17.551036 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21369021-706c-4fdc-9d1b-404a0994b1f5","Type":"ContainerDied","Data":"b9a091d909447e385c472c60b76ce00e13b3d49d70299839eccbe194831f7349"} Sep 10 13:23:17.551476 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:17.551105 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21369021-706c-4fdc-9d1b-404a0994b1f5","Type":"ContainerDied","Data":"8c7ccae57ac9c27e6a141e6074a3dd21a85da6865eb6b48f75162380be4dfa3f"} Sep 10 13:23:17.551476 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:17.551131 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21369021-706c-4fdc-9d1b-404a0994b1f5","Type":"ContainerDied","Data":"c51dccbb25ffcc1549bf9268cf7b5f920d0b4f5e016774a370e3434534837134"} Sep 10 13:23:17.551476 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:17.551156 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21369021-706c-4fdc-9d1b-404a0994b1f5","Type":"ContainerDied","Data":"b298bbd91d08880a70085b578023f9d2e3de295e05170e60136b15fcb742abc7"} Sep 10 13:23:17.551476 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:17.551179 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21369021-706c-4fdc-9d1b-404a0994b1f5","Type":"ContainerDied","Data":"b3e37610262018c102dbb174b1be06d334ac55e64e036c5c7733facdf1927479"} Sep 10 13:23:18.034427 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.034376 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.117242 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.117204 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-cluster-tls-config\") pod \"21369021-706c-4fdc-9d1b-404a0994b1f5\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " Sep 10 13:23:18.117461 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.117303 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-secret-alertmanager-kube-rbac-proxy-web\") pod \"21369021-706c-4fdc-9d1b-404a0994b1f5\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " Sep 10 13:23:18.117461 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.117336 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-config-volume\") pod \"21369021-706c-4fdc-9d1b-404a0994b1f5\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " Sep 10 13:23:18.117461 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.117370 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/21369021-706c-4fdc-9d1b-404a0994b1f5-alertmanager-main-db\") pod \"21369021-706c-4fdc-9d1b-404a0994b1f5\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " Sep 10 13:23:18.117461 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.117411 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21369021-706c-4fdc-9d1b-404a0994b1f5-alertmanager-trusted-ca-bundle\") pod \"21369021-706c-4fdc-9d1b-404a0994b1f5\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " Sep 10 13:23:18.117461 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.117438 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/21369021-706c-4fdc-9d1b-404a0994b1f5-tls-assets\") pod \"21369021-706c-4fdc-9d1b-404a0994b1f5\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " Sep 10 13:23:18.117759 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.117467 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21369021-706c-4fdc-9d1b-404a0994b1f5-metrics-client-ca\") pod \"21369021-706c-4fdc-9d1b-404a0994b1f5\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " Sep 10 13:23:18.117759 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.117493 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4xzf\" (UniqueName: \"kubernetes.io/projected/21369021-706c-4fdc-9d1b-404a0994b1f5-kube-api-access-d4xzf\") pod \"21369021-706c-4fdc-9d1b-404a0994b1f5\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " Sep 10 13:23:18.117759 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.117521 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-secret-alertmanager-main-tls\") pod \"21369021-706c-4fdc-9d1b-404a0994b1f5\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " Sep 10 13:23:18.117759 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.117572 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-web-config\") pod \"21369021-706c-4fdc-9d1b-404a0994b1f5\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " Sep 10 13:23:18.117759 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.117597 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"21369021-706c-4fdc-9d1b-404a0994b1f5\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " Sep 10 13:23:18.117759 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.117622 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/21369021-706c-4fdc-9d1b-404a0994b1f5-config-out\") pod \"21369021-706c-4fdc-9d1b-404a0994b1f5\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " Sep 10 13:23:18.117759 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.117658 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-secret-alertmanager-kube-rbac-proxy\") pod \"21369021-706c-4fdc-9d1b-404a0994b1f5\" (UID: \"21369021-706c-4fdc-9d1b-404a0994b1f5\") " Sep 10 13:23:18.118117 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.117749 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21369021-706c-4fdc-9d1b-404a0994b1f5-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "21369021-706c-4fdc-9d1b-404a0994b1f5" (UID: "21369021-706c-4fdc-9d1b-404a0994b1f5"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:23:18.118117 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.117964 2560 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/21369021-706c-4fdc-9d1b-404a0994b1f5-alertmanager-main-db\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:23:18.118237 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.118110 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21369021-706c-4fdc-9d1b-404a0994b1f5-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "21369021-706c-4fdc-9d1b-404a0994b1f5" (UID: "21369021-706c-4fdc-9d1b-404a0994b1f5"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:23:18.118758 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.118294 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21369021-706c-4fdc-9d1b-404a0994b1f5-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "21369021-706c-4fdc-9d1b-404a0994b1f5" (UID: "21369021-706c-4fdc-9d1b-404a0994b1f5"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:23:18.120279 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.120237 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-config-volume" (OuterVolumeSpecName: "config-volume") pod "21369021-706c-4fdc-9d1b-404a0994b1f5" (UID: "21369021-706c-4fdc-9d1b-404a0994b1f5"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 13:23:18.120555 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.120527 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "21369021-706c-4fdc-9d1b-404a0994b1f5" (UID: "21369021-706c-4fdc-9d1b-404a0994b1f5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 13:23:18.120723 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.120700 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "21369021-706c-4fdc-9d1b-404a0994b1f5" (UID: "21369021-706c-4fdc-9d1b-404a0994b1f5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 13:23:18.121265 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.121243 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "21369021-706c-4fdc-9d1b-404a0994b1f5" (UID: "21369021-706c-4fdc-9d1b-404a0994b1f5"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 13:23:18.122276 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.122239 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "21369021-706c-4fdc-9d1b-404a0994b1f5" (UID: "21369021-706c-4fdc-9d1b-404a0994b1f5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 13:23:18.122276 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.122255 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21369021-706c-4fdc-9d1b-404a0994b1f5-kube-api-access-d4xzf" (OuterVolumeSpecName: "kube-api-access-d4xzf") pod "21369021-706c-4fdc-9d1b-404a0994b1f5" (UID: "21369021-706c-4fdc-9d1b-404a0994b1f5"). InnerVolumeSpecName "kube-api-access-d4xzf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:23:18.122460 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.122371 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21369021-706c-4fdc-9d1b-404a0994b1f5-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "21369021-706c-4fdc-9d1b-404a0994b1f5" (UID: "21369021-706c-4fdc-9d1b-404a0994b1f5"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:23:18.122656 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.122621 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21369021-706c-4fdc-9d1b-404a0994b1f5-config-out" (OuterVolumeSpecName: "config-out") pod "21369021-706c-4fdc-9d1b-404a0994b1f5" (UID: "21369021-706c-4fdc-9d1b-404a0994b1f5"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:23:18.125788 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.125758 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "21369021-706c-4fdc-9d1b-404a0994b1f5" (UID: "21369021-706c-4fdc-9d1b-404a0994b1f5"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 13:23:18.132598 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.132572 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-web-config" (OuterVolumeSpecName: "web-config") pod "21369021-706c-4fdc-9d1b-404a0994b1f5" (UID: "21369021-706c-4fdc-9d1b-404a0994b1f5"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 13:23:18.219135 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.219079 2560 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-config-volume\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:23:18.219135 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.219132 2560 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21369021-706c-4fdc-9d1b-404a0994b1f5-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:23:18.219135 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.219146 2560 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/21369021-706c-4fdc-9d1b-404a0994b1f5-tls-assets\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:23:18.219449 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.219162 2560 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21369021-706c-4fdc-9d1b-404a0994b1f5-metrics-client-ca\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:23:18.219449 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.219173 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4xzf\" (UniqueName: \"kubernetes.io/projected/21369021-706c-4fdc-9d1b-404a0994b1f5-kube-api-access-d4xzf\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:23:18.219449 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.219186 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-secret-alertmanager-main-tls\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:23:18.219449 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.219222 2560 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-web-config\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:23:18.219449 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.219236 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:23:18.219449 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.219247 2560 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/21369021-706c-4fdc-9d1b-404a0994b1f5-config-out\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:23:18.219449 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.219258 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:23:18.219449 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.219269 2560 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-cluster-tls-config\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:23:18.219449 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.219281 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/21369021-706c-4fdc-9d1b-404a0994b1f5-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:23:18.558228 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.558197 2560 generic.go:356] "Generic (PLEG): container finished" podID="21369021-706c-4fdc-9d1b-404a0994b1f5" containerID="86114bbcd75b5c080391e5c8fd632f2d1fa0e37cab2a730bb52e32526c04d565" exitCode=0 Sep 10 13:23:18.558624 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.558298 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.561099 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.561064 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21369021-706c-4fdc-9d1b-404a0994b1f5","Type":"ContainerDied","Data":"86114bbcd75b5c080391e5c8fd632f2d1fa0e37cab2a730bb52e32526c04d565"} Sep 10 13:23:18.561254 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.561138 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21369021-706c-4fdc-9d1b-404a0994b1f5","Type":"ContainerDied","Data":"da7478386c11ea36ff719ad35203571b3d8bcbb0d9f36f19c9088023e3c2c96d"} Sep 10 13:23:18.561254 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.561209 2560 scope.go:117] "RemoveContainer" containerID="b9a091d909447e385c472c60b76ce00e13b3d49d70299839eccbe194831f7349" Sep 10 13:23:18.569777 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.569757 2560 scope.go:117] "RemoveContainer" containerID="8c7ccae57ac9c27e6a141e6074a3dd21a85da6865eb6b48f75162380be4dfa3f" Sep 10 13:23:18.577021 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.577004 2560 scope.go:117] "RemoveContainer" containerID="c51dccbb25ffcc1549bf9268cf7b5f920d0b4f5e016774a370e3434534837134" Sep 10 13:23:18.583800 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.583781 2560 scope.go:117] "RemoveContainer" containerID="86114bbcd75b5c080391e5c8fd632f2d1fa0e37cab2a730bb52e32526c04d565" Sep 10 13:23:18.587586 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.587564 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 13:23:18.591180 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.591163 2560 scope.go:117] "RemoveContainer" containerID="b298bbd91d08880a70085b578023f9d2e3de295e05170e60136b15fcb742abc7" Sep 10 13:23:18.592335 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.592317 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 13:23:18.598630 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.598613 2560 scope.go:117] "RemoveContainer" containerID="b3e37610262018c102dbb174b1be06d334ac55e64e036c5c7733facdf1927479" Sep 10 13:23:18.605997 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.605980 2560 scope.go:117] "RemoveContainer" containerID="44a8848183abe6c9a26263201814dde6874c10b0d2057d7cadd85691551c1853" Sep 10 13:23:18.613176 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.613156 2560 scope.go:117] "RemoveContainer" containerID="b9a091d909447e385c472c60b76ce00e13b3d49d70299839eccbe194831f7349" Sep 10 13:23:18.613505 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:23:18.613483 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9a091d909447e385c472c60b76ce00e13b3d49d70299839eccbe194831f7349\": container with ID starting with b9a091d909447e385c472c60b76ce00e13b3d49d70299839eccbe194831f7349 not found: ID does not exist" containerID="b9a091d909447e385c472c60b76ce00e13b3d49d70299839eccbe194831f7349" Sep 10 13:23:18.613611 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.613512 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9a091d909447e385c472c60b76ce00e13b3d49d70299839eccbe194831f7349"} err="failed to get container status \"b9a091d909447e385c472c60b76ce00e13b3d49d70299839eccbe194831f7349\": rpc error: code = NotFound desc = could not find container \"b9a091d909447e385c472c60b76ce00e13b3d49d70299839eccbe194831f7349\": container with ID starting with b9a091d909447e385c472c60b76ce00e13b3d49d70299839eccbe194831f7349 not found: ID does not exist" Sep 10 13:23:18.613611 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.613532 2560 scope.go:117] "RemoveContainer" containerID="8c7ccae57ac9c27e6a141e6074a3dd21a85da6865eb6b48f75162380be4dfa3f" Sep 10 13:23:18.613788 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:23:18.613770 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c7ccae57ac9c27e6a141e6074a3dd21a85da6865eb6b48f75162380be4dfa3f\": container with ID starting with 8c7ccae57ac9c27e6a141e6074a3dd21a85da6865eb6b48f75162380be4dfa3f not found: ID does not exist" containerID="8c7ccae57ac9c27e6a141e6074a3dd21a85da6865eb6b48f75162380be4dfa3f" Sep 10 13:23:18.613827 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.613795 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c7ccae57ac9c27e6a141e6074a3dd21a85da6865eb6b48f75162380be4dfa3f"} err="failed to get container status \"8c7ccae57ac9c27e6a141e6074a3dd21a85da6865eb6b48f75162380be4dfa3f\": rpc error: code = NotFound desc = could not find container \"8c7ccae57ac9c27e6a141e6074a3dd21a85da6865eb6b48f75162380be4dfa3f\": container with ID starting with 8c7ccae57ac9c27e6a141e6074a3dd21a85da6865eb6b48f75162380be4dfa3f not found: ID does not exist" Sep 10 13:23:18.613827 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.613811 2560 scope.go:117] "RemoveContainer" containerID="c51dccbb25ffcc1549bf9268cf7b5f920d0b4f5e016774a370e3434534837134" Sep 10 13:23:18.614065 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:23:18.614048 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c51dccbb25ffcc1549bf9268cf7b5f920d0b4f5e016774a370e3434534837134\": container with ID starting with c51dccbb25ffcc1549bf9268cf7b5f920d0b4f5e016774a370e3434534837134 not found: ID does not exist" containerID="c51dccbb25ffcc1549bf9268cf7b5f920d0b4f5e016774a370e3434534837134" Sep 10 13:23:18.614110 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.614070 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c51dccbb25ffcc1549bf9268cf7b5f920d0b4f5e016774a370e3434534837134"} err="failed to get container status \"c51dccbb25ffcc1549bf9268cf7b5f920d0b4f5e016774a370e3434534837134\": rpc error: code = NotFound desc = could not find container \"c51dccbb25ffcc1549bf9268cf7b5f920d0b4f5e016774a370e3434534837134\": container with ID starting with c51dccbb25ffcc1549bf9268cf7b5f920d0b4f5e016774a370e3434534837134 not found: ID does not exist" Sep 10 13:23:18.614110 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.614084 2560 scope.go:117] "RemoveContainer" containerID="86114bbcd75b5c080391e5c8fd632f2d1fa0e37cab2a730bb52e32526c04d565" Sep 10 13:23:18.614297 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:23:18.614281 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86114bbcd75b5c080391e5c8fd632f2d1fa0e37cab2a730bb52e32526c04d565\": container with ID starting with 86114bbcd75b5c080391e5c8fd632f2d1fa0e37cab2a730bb52e32526c04d565 not found: ID does not exist" containerID="86114bbcd75b5c080391e5c8fd632f2d1fa0e37cab2a730bb52e32526c04d565" Sep 10 13:23:18.614344 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.614299 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86114bbcd75b5c080391e5c8fd632f2d1fa0e37cab2a730bb52e32526c04d565"} err="failed to get container status \"86114bbcd75b5c080391e5c8fd632f2d1fa0e37cab2a730bb52e32526c04d565\": rpc error: code = NotFound desc = could not find container \"86114bbcd75b5c080391e5c8fd632f2d1fa0e37cab2a730bb52e32526c04d565\": container with ID starting with 86114bbcd75b5c080391e5c8fd632f2d1fa0e37cab2a730bb52e32526c04d565 not found: ID does not exist" Sep 10 13:23:18.614344 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.614311 2560 scope.go:117] "RemoveContainer" containerID="b298bbd91d08880a70085b578023f9d2e3de295e05170e60136b15fcb742abc7" Sep 10 13:23:18.614702 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:23:18.614676 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b298bbd91d08880a70085b578023f9d2e3de295e05170e60136b15fcb742abc7\": container with ID starting with b298bbd91d08880a70085b578023f9d2e3de295e05170e60136b15fcb742abc7 not found: ID does not exist" containerID="b298bbd91d08880a70085b578023f9d2e3de295e05170e60136b15fcb742abc7" Sep 10 13:23:18.614702 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.614704 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b298bbd91d08880a70085b578023f9d2e3de295e05170e60136b15fcb742abc7"} err="failed to get container status \"b298bbd91d08880a70085b578023f9d2e3de295e05170e60136b15fcb742abc7\": rpc error: code = NotFound desc = could not find container \"b298bbd91d08880a70085b578023f9d2e3de295e05170e60136b15fcb742abc7\": container with ID starting with b298bbd91d08880a70085b578023f9d2e3de295e05170e60136b15fcb742abc7 not found: ID does not exist" Sep 10 13:23:18.614843 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.614719 2560 scope.go:117] "RemoveContainer" containerID="b3e37610262018c102dbb174b1be06d334ac55e64e036c5c7733facdf1927479" Sep 10 13:23:18.615106 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:23:18.614987 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3e37610262018c102dbb174b1be06d334ac55e64e036c5c7733facdf1927479\": container with ID starting with b3e37610262018c102dbb174b1be06d334ac55e64e036c5c7733facdf1927479 not found: ID does not exist" containerID="b3e37610262018c102dbb174b1be06d334ac55e64e036c5c7733facdf1927479" Sep 10 13:23:18.615106 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.615016 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3e37610262018c102dbb174b1be06d334ac55e64e036c5c7733facdf1927479"} err="failed to get container status \"b3e37610262018c102dbb174b1be06d334ac55e64e036c5c7733facdf1927479\": rpc error: code = NotFound desc = could not find container \"b3e37610262018c102dbb174b1be06d334ac55e64e036c5c7733facdf1927479\": container with ID starting with b3e37610262018c102dbb174b1be06d334ac55e64e036c5c7733facdf1927479 not found: ID does not exist" Sep 10 13:23:18.615106 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.615041 2560 scope.go:117] "RemoveContainer" containerID="44a8848183abe6c9a26263201814dde6874c10b0d2057d7cadd85691551c1853" Sep 10 13:23:18.615360 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:23:18.615331 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44a8848183abe6c9a26263201814dde6874c10b0d2057d7cadd85691551c1853\": container with ID starting with 44a8848183abe6c9a26263201814dde6874c10b0d2057d7cadd85691551c1853 not found: ID does not exist" containerID="44a8848183abe6c9a26263201814dde6874c10b0d2057d7cadd85691551c1853" Sep 10 13:23:18.615445 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.615363 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44a8848183abe6c9a26263201814dde6874c10b0d2057d7cadd85691551c1853"} err="failed to get container status \"44a8848183abe6c9a26263201814dde6874c10b0d2057d7cadd85691551c1853\": rpc error: code = NotFound desc = could not find container \"44a8848183abe6c9a26263201814dde6874c10b0d2057d7cadd85691551c1853\": container with ID starting with 44a8848183abe6c9a26263201814dde6874c10b0d2057d7cadd85691551c1853 not found: ID does not exist" Sep 10 13:23:18.625469 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.625440 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 13:23:18.625765 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.625749 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ef3618e-e54b-46bf-8347-69b8e72720fd" containerName="registry" Sep 10 13:23:18.625843 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.625769 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ef3618e-e54b-46bf-8347-69b8e72720fd" containerName="registry" Sep 10 13:23:18.625843 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.625781 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21369021-706c-4fdc-9d1b-404a0994b1f5" containerName="prom-label-proxy" Sep 10 13:23:18.625843 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.625788 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="21369021-706c-4fdc-9d1b-404a0994b1f5" containerName="prom-label-proxy" Sep 10 13:23:18.625843 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.625803 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21369021-706c-4fdc-9d1b-404a0994b1f5" containerName="init-config-reloader" Sep 10 13:23:18.625843 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.625812 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="21369021-706c-4fdc-9d1b-404a0994b1f5" containerName="init-config-reloader" Sep 10 13:23:18.625843 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.625822 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21369021-706c-4fdc-9d1b-404a0994b1f5" containerName="config-reloader" Sep 10 13:23:18.625843 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.625829 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="21369021-706c-4fdc-9d1b-404a0994b1f5" containerName="config-reloader" Sep 10 13:23:18.625843 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.625844 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21369021-706c-4fdc-9d1b-404a0994b1f5" containerName="kube-rbac-proxy-metric" Sep 10 13:23:18.626250 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.625852 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="21369021-706c-4fdc-9d1b-404a0994b1f5" containerName="kube-rbac-proxy-metric" Sep 10 13:23:18.626250 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.625868 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21369021-706c-4fdc-9d1b-404a0994b1f5" containerName="kube-rbac-proxy" Sep 10 13:23:18.626250 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.625877 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="21369021-706c-4fdc-9d1b-404a0994b1f5" containerName="kube-rbac-proxy" Sep 10 13:23:18.626250 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.625893 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21369021-706c-4fdc-9d1b-404a0994b1f5" containerName="alertmanager" Sep 10 13:23:18.626250 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.625901 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="21369021-706c-4fdc-9d1b-404a0994b1f5" containerName="alertmanager" Sep 10 13:23:18.626250 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.625918 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21369021-706c-4fdc-9d1b-404a0994b1f5" containerName="kube-rbac-proxy-web" Sep 10 13:23:18.626250 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.625926 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="21369021-706c-4fdc-9d1b-404a0994b1f5" containerName="kube-rbac-proxy-web" Sep 10 13:23:18.626250 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.625994 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="21369021-706c-4fdc-9d1b-404a0994b1f5" containerName="kube-rbac-proxy-web" Sep 10 13:23:18.626250 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.626006 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="21369021-706c-4fdc-9d1b-404a0994b1f5" containerName="kube-rbac-proxy" Sep 10 13:23:18.626250 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.626015 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ef3618e-e54b-46bf-8347-69b8e72720fd" containerName="registry" Sep 10 13:23:18.626250 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.626025 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="21369021-706c-4fdc-9d1b-404a0994b1f5" containerName="alertmanager" Sep 10 13:23:18.626250 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.626037 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="21369021-706c-4fdc-9d1b-404a0994b1f5" containerName="kube-rbac-proxy-metric" Sep 10 13:23:18.626250 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.626048 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="21369021-706c-4fdc-9d1b-404a0994b1f5" containerName="config-reloader" Sep 10 13:23:18.626250 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.626056 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="21369021-706c-4fdc-9d1b-404a0994b1f5" containerName="prom-label-proxy" Sep 10 13:23:18.630155 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.630134 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.633171 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.633094 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-5dpvg\"" Sep 10 13:23:18.633171 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.633106 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 10 13:23:18.633171 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.633100 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 10 13:23:18.633171 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.633152 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 10 13:23:18.633452 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.633197 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 10 13:23:18.633452 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.633142 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 10 13:23:18.633452 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.633136 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 10 13:23:18.633452 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.633443 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 10 13:23:18.633710 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.633693 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 10 13:23:18.641605 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.641579 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 10 13:23:18.647281 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.647260 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 13:23:18.724961 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.724908 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.724961 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.724964 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.725232 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.725051 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-config-out\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.725232 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.725092 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.725232 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.725116 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-web-config\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.725232 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.725140 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.725232 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.725167 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.725232 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.725203 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.725533 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.725249 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwd4m\" (UniqueName: \"kubernetes.io/projected/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-kube-api-access-wwd4m\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.725533 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.725286 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.725533 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.725326 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-config-volume\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.725533 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.725366 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.725533 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.725390 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.826211 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.826093 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.826211 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.826166 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.826211 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.826193 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.826609 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.826226 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-config-out\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.826609 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.826249 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.826609 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.826387 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-web-config\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.826609 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.826453 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.826609 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.826490 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.826609 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.826533 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.826609 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.826567 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwd4m\" (UniqueName: \"kubernetes.io/projected/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-kube-api-access-wwd4m\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.826609 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.826577 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.827072 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.826633 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.827072 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.826689 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-config-volume\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.827072 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.826754 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.827072 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.826974 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.827361 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.827335 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.829190 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.829165 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-config-out\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.829366 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.829312 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.829545 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.829520 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.829652 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.829524 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.829652 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.829604 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.830051 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.830028 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.830132 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.830076 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.830839 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.830814 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-web-config\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.830990 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.830974 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-config-volume\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.838985 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.838963 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwd4m\" (UniqueName: \"kubernetes.io/projected/6d221cc8-d9d7-49a6-82e4-14e719a4dc96-kube-api-access-wwd4m\") pod \"alertmanager-main-0\" (UID: \"6d221cc8-d9d7-49a6-82e4-14e719a4dc96\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:18.940834 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:18.940786 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 13:23:19.225709 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:19.225680 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 13:23:19.227805 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:23:19.227779 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d221cc8_d9d7_49a6_82e4_14e719a4dc96.slice/crio-ea1783899ea5f06e72c28febff405ca835b4411281d0b554e672b6206459eeb9 WatchSource:0}: Error finding container ea1783899ea5f06e72c28febff405ca835b4411281d0b554e672b6206459eeb9: Status 404 returned error can't find the container with id ea1783899ea5f06e72c28febff405ca835b4411281d0b554e672b6206459eeb9 Sep 10 13:23:19.563951 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:19.563863 2560 generic.go:356] "Generic (PLEG): container finished" podID="6d221cc8-d9d7-49a6-82e4-14e719a4dc96" containerID="139a3e49f8306d16ac3c3fdfd069736fbd41fc01a50a2856f9554d739e42fc7e" exitCode=0 Sep 10 13:23:19.563951 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:19.563926 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6d221cc8-d9d7-49a6-82e4-14e719a4dc96","Type":"ContainerDied","Data":"139a3e49f8306d16ac3c3fdfd069736fbd41fc01a50a2856f9554d739e42fc7e"} Sep 10 13:23:19.564412 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:19.563985 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6d221cc8-d9d7-49a6-82e4-14e719a4dc96","Type":"ContainerStarted","Data":"ea1783899ea5f06e72c28febff405ca835b4411281d0b554e672b6206459eeb9"} Sep 10 13:23:20.567243 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:20.567212 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21369021-706c-4fdc-9d1b-404a0994b1f5" path="/var/lib/kubelet/pods/21369021-706c-4fdc-9d1b-404a0994b1f5/volumes" Sep 10 13:23:20.571418 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:20.571374 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6d221cc8-d9d7-49a6-82e4-14e719a4dc96","Type":"ContainerStarted","Data":"5f9d7874db11130357bca4d3c2b95403719b4267e6e5cd37cded4890d7c1c814"} Sep 10 13:23:20.571541 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:20.571452 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6d221cc8-d9d7-49a6-82e4-14e719a4dc96","Type":"ContainerStarted","Data":"d598e33f1ad51049544eb3ca38f659848e5c501101c7e9b9f936e6acf2b20885"} Sep 10 13:23:20.571541 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:20.571490 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6d221cc8-d9d7-49a6-82e4-14e719a4dc96","Type":"ContainerStarted","Data":"916cf5d5d2ef817dc56bfd25af171defe7f4830860bad17decc25651a669282a"} Sep 10 13:23:20.571541 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:20.571517 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6d221cc8-d9d7-49a6-82e4-14e719a4dc96","Type":"ContainerStarted","Data":"99e7ee70f4ed350bb1accea07a92ed49a233ddf4d1ffa0baccb12bf29afc1672"} Sep 10 13:23:20.571541 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:20.571540 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6d221cc8-d9d7-49a6-82e4-14e719a4dc96","Type":"ContainerStarted","Data":"cdb899c76311f3f4f4ce6bd611275ea22f17742d96961c1e503241ffa1447423"} Sep 10 13:23:20.571704 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:20.571575 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6d221cc8-d9d7-49a6-82e4-14e719a4dc96","Type":"ContainerStarted","Data":"4698a7b75308ba5728732f3819fce193015e066b631a8a1529293880632dace7"} Sep 10 13:23:20.609861 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:20.609806 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.593341123 podStartE2EDuration="2.609789325s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.448202ms" isStatefulPod=false podCreationTimestamp="2026-09-10 13:23:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:23:20.608817382 +0000 UTC m=+310.503534801" watchObservedRunningTime="2026-09-10 13:23:20.609789325 +0000 UTC m=+310.504506742" Sep 10 13:23:24.944960 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:24.944909 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:23:25.057343 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:25.057304 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:23:36.044304 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:36.044262 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:23:36.157372 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:23:36.157340 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:24:07.369596 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:24:07.369512 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:24:07.463122 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:24:07.463083 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:24:14.103711 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:24:14.103666 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:24:14.248139 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:24:14.248097 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:24:31.012467 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:24:31.012427 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:24:31.084352 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:24:31.084313 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:24:43.021641 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:24:43.021599 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:24:43.101587 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:24:43.101554 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:24:44.730959 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:24:44.730920 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:24:44.789611 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:24:44.789574 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:24:51.886743 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:24:51.886699 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:24:52.923087 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:24:52.923055 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:24:53.024083 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:24:53.024039 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:24:56.165545 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:24:56.165508 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:24:56.259844 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:24:56.259811 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:24:56.322471 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:24:56.322432 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:25:32.668628 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:32.668589 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:25:33.492630 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:33.492593 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m49r"] Sep 10 13:25:33.495895 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:33.495877 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m49r" Sep 10 13:25:33.498556 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:33.498534 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 10 13:25:33.499345 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:33.499330 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 10 13:25:33.499345 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:33.499337 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-v2fn8\"" Sep 10 13:25:33.510888 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:33.510858 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m49r"] Sep 10 13:25:33.570910 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:33.570868 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dd91261a-13f4-496c-aea9-892fae8116e3-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m49r\" (UID: \"dd91261a-13f4-496c-aea9-892fae8116e3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m49r" Sep 10 13:25:33.571101 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:33.570920 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2jjv\" (UniqueName: \"kubernetes.io/projected/dd91261a-13f4-496c-aea9-892fae8116e3-kube-api-access-j2jjv\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m49r\" (UID: \"dd91261a-13f4-496c-aea9-892fae8116e3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m49r" Sep 10 13:25:33.571101 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:33.570944 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dd91261a-13f4-496c-aea9-892fae8116e3-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m49r\" (UID: \"dd91261a-13f4-496c-aea9-892fae8116e3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m49r" Sep 10 13:25:33.671358 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:33.671320 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dd91261a-13f4-496c-aea9-892fae8116e3-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m49r\" (UID: \"dd91261a-13f4-496c-aea9-892fae8116e3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m49r" Sep 10 13:25:33.671782 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:33.671371 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2jjv\" (UniqueName: \"kubernetes.io/projected/dd91261a-13f4-496c-aea9-892fae8116e3-kube-api-access-j2jjv\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m49r\" (UID: \"dd91261a-13f4-496c-aea9-892fae8116e3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m49r" Sep 10 13:25:33.671782 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:33.671490 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dd91261a-13f4-496c-aea9-892fae8116e3-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m49r\" (UID: \"dd91261a-13f4-496c-aea9-892fae8116e3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m49r" Sep 10 13:25:33.671782 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:33.671778 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dd91261a-13f4-496c-aea9-892fae8116e3-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m49r\" (UID: \"dd91261a-13f4-496c-aea9-892fae8116e3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m49r" Sep 10 13:25:33.671908 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:33.671808 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dd91261a-13f4-496c-aea9-892fae8116e3-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m49r\" (UID: \"dd91261a-13f4-496c-aea9-892fae8116e3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m49r" Sep 10 13:25:33.682586 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:33.682558 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2jjv\" (UniqueName: \"kubernetes.io/projected/dd91261a-13f4-496c-aea9-892fae8116e3-kube-api-access-j2jjv\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m49r\" (UID: \"dd91261a-13f4-496c-aea9-892fae8116e3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m49r" Sep 10 13:25:33.804872 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:33.804777 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m49r" Sep 10 13:25:34.082832 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:34.082636 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m49r"] Sep 10 13:25:34.085340 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:25:34.085308 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd91261a_13f4_496c_aea9_892fae8116e3.slice/crio-fa7a6796b4fe66167255b20ce3fa07dd3b2e67803ae2c2e9bcf219764d01605c WatchSource:0}: Error finding container fa7a6796b4fe66167255b20ce3fa07dd3b2e67803ae2c2e9bcf219764d01605c: Status 404 returned error can't find the container with id fa7a6796b4fe66167255b20ce3fa07dd3b2e67803ae2c2e9bcf219764d01605c Sep 10 13:25:34.087331 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:34.087314 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 13:25:35.080533 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:35.080501 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m49r" event={"ID":"dd91261a-13f4-496c-aea9-892fae8116e3","Type":"ContainerStarted","Data":"fa7a6796b4fe66167255b20ce3fa07dd3b2e67803ae2c2e9bcf219764d01605c"} Sep 10 13:25:36.599798 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:36.599760 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:25:42.113415 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:42.113375 2560 generic.go:356] "Generic (PLEG): container finished" podID="dd91261a-13f4-496c-aea9-892fae8116e3" containerID="160c3e49faaa537451a56034955fc0b97d8180f5f21f7e327d3ed7a13e20c007" exitCode=0 Sep 10 13:25:42.113833 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:42.113465 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m49r" event={"ID":"dd91261a-13f4-496c-aea9-892fae8116e3","Type":"ContainerDied","Data":"160c3e49faaa537451a56034955fc0b97d8180f5f21f7e327d3ed7a13e20c007"} Sep 10 13:25:45.125678 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:45.125641 2560 generic.go:356] "Generic (PLEG): container finished" podID="dd91261a-13f4-496c-aea9-892fae8116e3" containerID="3fdd8d727545f8587657e1bde87f4378a4801e71b7a46c8fbc1d1802d16d7ed7" exitCode=0 Sep 10 13:25:45.126061 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:45.125714 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m49r" event={"ID":"dd91261a-13f4-496c-aea9-892fae8116e3","Type":"ContainerDied","Data":"3fdd8d727545f8587657e1bde87f4378a4801e71b7a46c8fbc1d1802d16d7ed7"} Sep 10 13:25:53.160594 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:53.160555 2560 generic.go:356] "Generic (PLEG): container finished" podID="dd91261a-13f4-496c-aea9-892fae8116e3" containerID="a5207156f90d5637ceafb1224060adc50d4a3d2acb2c71316af02874ceaa1234" exitCode=0 Sep 10 13:25:53.160986 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:53.160641 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m49r" event={"ID":"dd91261a-13f4-496c-aea9-892fae8116e3","Type":"ContainerDied","Data":"a5207156f90d5637ceafb1224060adc50d4a3d2acb2c71316af02874ceaa1234"} Sep 10 13:25:54.296899 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:54.296868 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m49r" Sep 10 13:25:54.453270 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:54.453232 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2jjv\" (UniqueName: \"kubernetes.io/projected/dd91261a-13f4-496c-aea9-892fae8116e3-kube-api-access-j2jjv\") pod \"dd91261a-13f4-496c-aea9-892fae8116e3\" (UID: \"dd91261a-13f4-496c-aea9-892fae8116e3\") " Sep 10 13:25:54.453471 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:54.453305 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dd91261a-13f4-496c-aea9-892fae8116e3-bundle\") pod \"dd91261a-13f4-496c-aea9-892fae8116e3\" (UID: \"dd91261a-13f4-496c-aea9-892fae8116e3\") " Sep 10 13:25:54.453471 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:54.453326 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dd91261a-13f4-496c-aea9-892fae8116e3-util\") pod \"dd91261a-13f4-496c-aea9-892fae8116e3\" (UID: \"dd91261a-13f4-496c-aea9-892fae8116e3\") " Sep 10 13:25:54.454183 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:54.454150 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd91261a-13f4-496c-aea9-892fae8116e3-bundle" (OuterVolumeSpecName: "bundle") pod "dd91261a-13f4-496c-aea9-892fae8116e3" (UID: "dd91261a-13f4-496c-aea9-892fae8116e3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:25:54.455452 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:54.455430 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd91261a-13f4-496c-aea9-892fae8116e3-kube-api-access-j2jjv" (OuterVolumeSpecName: "kube-api-access-j2jjv") pod "dd91261a-13f4-496c-aea9-892fae8116e3" (UID: "dd91261a-13f4-496c-aea9-892fae8116e3"). InnerVolumeSpecName "kube-api-access-j2jjv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:25:54.459119 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:54.459093 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd91261a-13f4-496c-aea9-892fae8116e3-util" (OuterVolumeSpecName: "util") pod "dd91261a-13f4-496c-aea9-892fae8116e3" (UID: "dd91261a-13f4-496c-aea9-892fae8116e3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:25:54.554619 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:54.554581 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2jjv\" (UniqueName: \"kubernetes.io/projected/dd91261a-13f4-496c-aea9-892fae8116e3-kube-api-access-j2jjv\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:25:54.554619 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:54.554616 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dd91261a-13f4-496c-aea9-892fae8116e3-bundle\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:25:54.554619 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:54.554626 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dd91261a-13f4-496c-aea9-892fae8116e3-util\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:25:55.170138 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:55.170112 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m49r" Sep 10 13:25:55.170305 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:55.170108 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409m49r" event={"ID":"dd91261a-13f4-496c-aea9-892fae8116e3","Type":"ContainerDied","Data":"fa7a6796b4fe66167255b20ce3fa07dd3b2e67803ae2c2e9bcf219764d01605c"} Sep 10 13:25:55.170305 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:55.170230 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fa7a6796b4fe66167255b20ce3fa07dd3b2e67803ae2c2e9bcf219764d01605c" Sep 10 13:25:59.883301 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:25:59.883266 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:00.237415 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:00.237374 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:02.136855 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:02.136818 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k77bl"] Sep 10 13:26:02.137234 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:02.137127 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd91261a-13f4-496c-aea9-892fae8116e3" containerName="util" Sep 10 13:26:02.137234 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:02.137138 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd91261a-13f4-496c-aea9-892fae8116e3" containerName="util" Sep 10 13:26:02.137234 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:02.137154 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd91261a-13f4-496c-aea9-892fae8116e3" containerName="pull" Sep 10 13:26:02.137234 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:02.137159 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd91261a-13f4-496c-aea9-892fae8116e3" containerName="pull" Sep 10 13:26:02.137234 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:02.137164 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd91261a-13f4-496c-aea9-892fae8116e3" containerName="extract" Sep 10 13:26:02.137234 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:02.137171 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd91261a-13f4-496c-aea9-892fae8116e3" containerName="extract" Sep 10 13:26:02.137234 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:02.137228 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd91261a-13f4-496c-aea9-892fae8116e3" containerName="extract" Sep 10 13:26:02.141613 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:02.141591 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k77bl" Sep 10 13:26:02.146177 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:02.146157 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Sep 10 13:26:02.146302 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:02.146243 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Sep 10 13:26:02.146520 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:02.146507 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-2lxl8\"" Sep 10 13:26:02.165515 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:02.165480 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k77bl"] Sep 10 13:26:02.210739 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:02.210702 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ztmf\" (UniqueName: \"kubernetes.io/projected/27229e63-07d1-4031-8710-5104f9cc618d-kube-api-access-8ztmf\") pod \"cert-manager-operator-controller-manager-55c444bf86-k77bl\" (UID: \"27229e63-07d1-4031-8710-5104f9cc618d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k77bl" Sep 10 13:26:02.210739 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:02.210737 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/27229e63-07d1-4031-8710-5104f9cc618d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-k77bl\" (UID: \"27229e63-07d1-4031-8710-5104f9cc618d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k77bl" Sep 10 13:26:02.311859 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:02.311828 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ztmf\" (UniqueName: \"kubernetes.io/projected/27229e63-07d1-4031-8710-5104f9cc618d-kube-api-access-8ztmf\") pod \"cert-manager-operator-controller-manager-55c444bf86-k77bl\" (UID: \"27229e63-07d1-4031-8710-5104f9cc618d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k77bl" Sep 10 13:26:02.311859 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:02.311860 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/27229e63-07d1-4031-8710-5104f9cc618d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-k77bl\" (UID: \"27229e63-07d1-4031-8710-5104f9cc618d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k77bl" Sep 10 13:26:02.312204 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:02.312188 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/27229e63-07d1-4031-8710-5104f9cc618d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-k77bl\" (UID: \"27229e63-07d1-4031-8710-5104f9cc618d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k77bl" Sep 10 13:26:02.351803 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:02.351772 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ztmf\" (UniqueName: \"kubernetes.io/projected/27229e63-07d1-4031-8710-5104f9cc618d-kube-api-access-8ztmf\") pod \"cert-manager-operator-controller-manager-55c444bf86-k77bl\" (UID: \"27229e63-07d1-4031-8710-5104f9cc618d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k77bl" Sep 10 13:26:02.450693 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:02.450644 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k77bl" Sep 10 13:26:02.729821 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:02.729793 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k77bl"] Sep 10 13:26:02.732334 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:26:02.732300 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27229e63_07d1_4031_8710_5104f9cc618d.slice/crio-175bb0ab4e83371738e2691f09d2a98ca9c42c575bbfe0964d024d9bbd644a1e WatchSource:0}: Error finding container 175bb0ab4e83371738e2691f09d2a98ca9c42c575bbfe0964d024d9bbd644a1e: Status 404 returned error can't find the container with id 175bb0ab4e83371738e2691f09d2a98ca9c42c575bbfe0964d024d9bbd644a1e Sep 10 13:26:03.206927 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:03.206891 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k77bl" event={"ID":"27229e63-07d1-4031-8710-5104f9cc618d","Type":"ContainerStarted","Data":"175bb0ab4e83371738e2691f09d2a98ca9c42c575bbfe0964d024d9bbd644a1e"} Sep 10 13:26:03.900555 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:03.900522 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:04.023824 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:04.023782 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:06.222107 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:06.222066 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k77bl" event={"ID":"27229e63-07d1-4031-8710-5104f9cc618d","Type":"ContainerStarted","Data":"cda39ce7ebc8c3f364d0364e09799f5954ecd6da848d6fd53af64d3c72d6a159"} Sep 10 13:26:06.265823 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:06.265766 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-k77bl" podStartSLOduration=1.732364122 podStartE2EDuration="4.265750254s" totalImagesPullingTime="2.533386132s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:26:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:26:06.265710351 +0000 UTC m=+476.160427769" watchObservedRunningTime="2026-09-10 13:26:06.265750254 +0000 UTC m=+476.160467653" Sep 10 13:26:06.269810 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:06.269782 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:06.369760 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:06.369729 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:06.677324 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:06.677292 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:06.851235 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:06.851199 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4454s"] Sep 10 13:26:06.853398 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:06.853381 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4454s" Sep 10 13:26:06.857261 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:06.857234 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-v2fn8\"" Sep 10 13:26:06.857435 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:06.857366 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 10 13:26:06.857612 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:06.857583 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 10 13:26:06.869120 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:06.869091 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:06.872392 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:06.872365 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4454s"] Sep 10 13:26:06.945261 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:06.945170 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f1fa6da-9293-4834-97d9-67aa0af302f2-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4454s\" (UID: \"4f1fa6da-9293-4834-97d9-67aa0af302f2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4454s" Sep 10 13:26:06.945261 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:06.945241 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f1fa6da-9293-4834-97d9-67aa0af302f2-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4454s\" (UID: \"4f1fa6da-9293-4834-97d9-67aa0af302f2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4454s" Sep 10 13:26:06.945494 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:06.945286 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvnft\" (UniqueName: \"kubernetes.io/projected/4f1fa6da-9293-4834-97d9-67aa0af302f2-kube-api-access-bvnft\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4454s\" (UID: \"4f1fa6da-9293-4834-97d9-67aa0af302f2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4454s" Sep 10 13:26:07.046251 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:07.046208 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvnft\" (UniqueName: \"kubernetes.io/projected/4f1fa6da-9293-4834-97d9-67aa0af302f2-kube-api-access-bvnft\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4454s\" (UID: \"4f1fa6da-9293-4834-97d9-67aa0af302f2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4454s" Sep 10 13:26:07.046466 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:07.046275 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f1fa6da-9293-4834-97d9-67aa0af302f2-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4454s\" (UID: \"4f1fa6da-9293-4834-97d9-67aa0af302f2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4454s" Sep 10 13:26:07.046466 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:07.046314 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f1fa6da-9293-4834-97d9-67aa0af302f2-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4454s\" (UID: \"4f1fa6da-9293-4834-97d9-67aa0af302f2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4454s" Sep 10 13:26:07.046710 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:07.046690 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f1fa6da-9293-4834-97d9-67aa0af302f2-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4454s\" (UID: \"4f1fa6da-9293-4834-97d9-67aa0af302f2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4454s" Sep 10 13:26:07.046750 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:07.046695 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f1fa6da-9293-4834-97d9-67aa0af302f2-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4454s\" (UID: \"4f1fa6da-9293-4834-97d9-67aa0af302f2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4454s" Sep 10 13:26:07.066033 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:07.066004 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvnft\" (UniqueName: \"kubernetes.io/projected/4f1fa6da-9293-4834-97d9-67aa0af302f2-kube-api-access-bvnft\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4454s\" (UID: \"4f1fa6da-9293-4834-97d9-67aa0af302f2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4454s" Sep 10 13:26:07.162778 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:07.162745 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4454s" Sep 10 13:26:07.448851 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:07.448818 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4454s"] Sep 10 13:26:07.451188 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:26:07.451158 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f1fa6da_9293_4834_97d9_67aa0af302f2.slice/crio-44ada9cbc0971cd931771bd98fe88989493dae3702b6cf950a3fd96b77bd5a90 WatchSource:0}: Error finding container 44ada9cbc0971cd931771bd98fe88989493dae3702b6cf950a3fd96b77bd5a90: Status 404 returned error can't find the container with id 44ada9cbc0971cd931771bd98fe88989493dae3702b6cf950a3fd96b77bd5a90 Sep 10 13:26:08.234323 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:08.234290 2560 generic.go:356] "Generic (PLEG): container finished" podID="4f1fa6da-9293-4834-97d9-67aa0af302f2" containerID="58bc7cf0065a67e3d45456f747aa969ffcc5b4c2b4fe7b1b6732292eb8b700c1" exitCode=0 Sep 10 13:26:08.234558 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:08.234351 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4454s" event={"ID":"4f1fa6da-9293-4834-97d9-67aa0af302f2","Type":"ContainerDied","Data":"58bc7cf0065a67e3d45456f747aa969ffcc5b4c2b4fe7b1b6732292eb8b700c1"} Sep 10 13:26:08.234558 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:08.234417 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4454s" event={"ID":"4f1fa6da-9293-4834-97d9-67aa0af302f2","Type":"ContainerStarted","Data":"44ada9cbc0971cd931771bd98fe88989493dae3702b6cf950a3fd96b77bd5a90"} Sep 10 13:26:08.682295 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:08.682263 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:08.834122 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:08.834088 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:09.199844 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:09.199805 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-r48ft"] Sep 10 13:26:09.203171 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:09.203149 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-r48ft" Sep 10 13:26:09.205826 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:09.205796 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Sep 10 13:26:09.206544 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:09.206524 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-jjntn\"" Sep 10 13:26:09.206658 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:09.206528 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Sep 10 13:26:09.213128 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:09.213097 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-r48ft"] Sep 10 13:26:09.264112 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:09.262436 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sw5hk\" (UniqueName: \"kubernetes.io/projected/f2b1a8f8-ee8e-457d-a1e6-5b27315e8ce2-kube-api-access-sw5hk\") pod \"cert-manager-cainjector-559cdd8c87-r48ft\" (UID: \"f2b1a8f8-ee8e-457d-a1e6-5b27315e8ce2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-r48ft" Sep 10 13:26:09.264112 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:09.262526 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f2b1a8f8-ee8e-457d-a1e6-5b27315e8ce2-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-r48ft\" (UID: \"f2b1a8f8-ee8e-457d-a1e6-5b27315e8ce2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-r48ft" Sep 10 13:26:09.363364 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:09.363311 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sw5hk\" (UniqueName: \"kubernetes.io/projected/f2b1a8f8-ee8e-457d-a1e6-5b27315e8ce2-kube-api-access-sw5hk\") pod \"cert-manager-cainjector-559cdd8c87-r48ft\" (UID: \"f2b1a8f8-ee8e-457d-a1e6-5b27315e8ce2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-r48ft" Sep 10 13:26:09.363548 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:09.363384 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f2b1a8f8-ee8e-457d-a1e6-5b27315e8ce2-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-r48ft\" (UID: \"f2b1a8f8-ee8e-457d-a1e6-5b27315e8ce2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-r48ft" Sep 10 13:26:09.375528 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:09.375496 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f2b1a8f8-ee8e-457d-a1e6-5b27315e8ce2-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-r48ft\" (UID: \"f2b1a8f8-ee8e-457d-a1e6-5b27315e8ce2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-r48ft" Sep 10 13:26:09.375785 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:09.375754 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sw5hk\" (UniqueName: \"kubernetes.io/projected/f2b1a8f8-ee8e-457d-a1e6-5b27315e8ce2-kube-api-access-sw5hk\") pod \"cert-manager-cainjector-559cdd8c87-r48ft\" (UID: \"f2b1a8f8-ee8e-457d-a1e6-5b27315e8ce2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-r48ft" Sep 10 13:26:09.520058 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:09.519944 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-r48ft" Sep 10 13:26:09.807862 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:09.807776 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-r48ft"] Sep 10 13:26:10.231569 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:10.231531 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:10.336306 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:10.336266 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:10.349063 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:26:10.349020 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf2b1a8f8_ee8e_457d_a1e6_5b27315e8ce2.slice/crio-d5239b2db0937f6d85a140fd764c214d2a2ca9198f45aa20f55e757c32a44699 WatchSource:0}: Error finding container d5239b2db0937f6d85a140fd764c214d2a2ca9198f45aa20f55e757c32a44699: Status 404 returned error can't find the container with id d5239b2db0937f6d85a140fd764c214d2a2ca9198f45aa20f55e757c32a44699 Sep 10 13:26:11.251855 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:11.251811 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-r48ft" event={"ID":"f2b1a8f8-ee8e-457d-a1e6-5b27315e8ce2","Type":"ContainerStarted","Data":"d5239b2db0937f6d85a140fd764c214d2a2ca9198f45aa20f55e757c32a44699"} Sep 10 13:26:11.253925 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:11.253894 2560 generic.go:356] "Generic (PLEG): container finished" podID="4f1fa6da-9293-4834-97d9-67aa0af302f2" containerID="f6cf81c976166e802329a02782a4c9b7b6f87a874b89cec11cbd118d5f1a025a" exitCode=0 Sep 10 13:26:11.254068 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:11.253974 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4454s" event={"ID":"4f1fa6da-9293-4834-97d9-67aa0af302f2","Type":"ContainerDied","Data":"f6cf81c976166e802329a02782a4c9b7b6f87a874b89cec11cbd118d5f1a025a"} Sep 10 13:26:12.261395 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:12.261360 2560 generic.go:356] "Generic (PLEG): container finished" podID="4f1fa6da-9293-4834-97d9-67aa0af302f2" containerID="96854fa08544b5574f36d654a4bafa907564be8b6ffbe8f19a9619010780eb81" exitCode=0 Sep 10 13:26:12.261840 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:12.261444 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4454s" event={"ID":"4f1fa6da-9293-4834-97d9-67aa0af302f2","Type":"ContainerDied","Data":"96854fa08544b5574f36d654a4bafa907564be8b6ffbe8f19a9619010780eb81"} Sep 10 13:26:13.267217 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:13.267123 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-r48ft" event={"ID":"f2b1a8f8-ee8e-457d-a1e6-5b27315e8ce2","Type":"ContainerStarted","Data":"9a4986cbaa35af5eb5fe5d351bc840a534434b08acf4f13633c728468f218f7a"} Sep 10 13:26:13.306876 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:13.306786 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-r48ft" podStartSLOduration=1.731601417 podStartE2EDuration="4.306761356s" totalImagesPullingTime="2.575159939s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:26:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:26:13.306275211 +0000 UTC m=+483.200992630" watchObservedRunningTime="2026-09-10 13:26:13.306761356 +0000 UTC m=+483.201478775" Sep 10 13:26:13.406366 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:13.406338 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4454s" Sep 10 13:26:13.498507 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:13.498473 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f1fa6da-9293-4834-97d9-67aa0af302f2-bundle\") pod \"4f1fa6da-9293-4834-97d9-67aa0af302f2\" (UID: \"4f1fa6da-9293-4834-97d9-67aa0af302f2\") " Sep 10 13:26:13.498699 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:13.498611 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f1fa6da-9293-4834-97d9-67aa0af302f2-util\") pod \"4f1fa6da-9293-4834-97d9-67aa0af302f2\" (UID: \"4f1fa6da-9293-4834-97d9-67aa0af302f2\") " Sep 10 13:26:13.498699 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:13.498647 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bvnft\" (UniqueName: \"kubernetes.io/projected/4f1fa6da-9293-4834-97d9-67aa0af302f2-kube-api-access-bvnft\") pod \"4f1fa6da-9293-4834-97d9-67aa0af302f2\" (UID: \"4f1fa6da-9293-4834-97d9-67aa0af302f2\") " Sep 10 13:26:13.498949 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:13.498921 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f1fa6da-9293-4834-97d9-67aa0af302f2-bundle" (OuterVolumeSpecName: "bundle") pod "4f1fa6da-9293-4834-97d9-67aa0af302f2" (UID: "4f1fa6da-9293-4834-97d9-67aa0af302f2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:26:13.500749 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:13.500726 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f1fa6da-9293-4834-97d9-67aa0af302f2-kube-api-access-bvnft" (OuterVolumeSpecName: "kube-api-access-bvnft") pod "4f1fa6da-9293-4834-97d9-67aa0af302f2" (UID: "4f1fa6da-9293-4834-97d9-67aa0af302f2"). InnerVolumeSpecName "kube-api-access-bvnft". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:26:13.502971 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:13.502949 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f1fa6da-9293-4834-97d9-67aa0af302f2-util" (OuterVolumeSpecName: "util") pod "4f1fa6da-9293-4834-97d9-67aa0af302f2" (UID: "4f1fa6da-9293-4834-97d9-67aa0af302f2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:26:13.599511 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:13.599393 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvnft\" (UniqueName: \"kubernetes.io/projected/4f1fa6da-9293-4834-97d9-67aa0af302f2-kube-api-access-bvnft\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:26:13.599511 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:13.599452 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f1fa6da-9293-4834-97d9-67aa0af302f2-bundle\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:26:13.599511 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:13.599462 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f1fa6da-9293-4834-97d9-67aa0af302f2-util\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:26:13.976110 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:13.976077 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:14.073131 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:14.073100 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:14.273928 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:14.273839 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4454s" Sep 10 13:26:14.273928 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:14.273850 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4454s" event={"ID":"4f1fa6da-9293-4834-97d9-67aa0af302f2","Type":"ContainerDied","Data":"44ada9cbc0971cd931771bd98fe88989493dae3702b6cf950a3fd96b77bd5a90"} Sep 10 13:26:14.273928 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:14.273899 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="44ada9cbc0971cd931771bd98fe88989493dae3702b6cf950a3fd96b77bd5a90" Sep 10 13:26:16.096758 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:16.096727 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:18.155103 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:18.155068 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rz72f"] Sep 10 13:26:18.155490 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:18.155365 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f1fa6da-9293-4834-97d9-67aa0af302f2" containerName="pull" Sep 10 13:26:18.155490 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:18.155377 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f1fa6da-9293-4834-97d9-67aa0af302f2" containerName="pull" Sep 10 13:26:18.155490 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:18.155394 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f1fa6da-9293-4834-97d9-67aa0af302f2" containerName="extract" Sep 10 13:26:18.155490 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:18.155415 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f1fa6da-9293-4834-97d9-67aa0af302f2" containerName="extract" Sep 10 13:26:18.155490 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:18.155435 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f1fa6da-9293-4834-97d9-67aa0af302f2" containerName="util" Sep 10 13:26:18.155490 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:18.155443 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f1fa6da-9293-4834-97d9-67aa0af302f2" containerName="util" Sep 10 13:26:18.155711 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:18.155506 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f1fa6da-9293-4834-97d9-67aa0af302f2" containerName="extract" Sep 10 13:26:18.159060 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:18.159041 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-rz72f" Sep 10 13:26:18.161589 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:18.161566 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-5h5zr\"" Sep 10 13:26:18.169619 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:18.169594 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rz72f"] Sep 10 13:26:18.234260 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:18.234214 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnqpm\" (UniqueName: \"kubernetes.io/projected/a4b96d6e-b281-4c96-b186-43813038ba1e-kube-api-access-wnqpm\") pod \"cert-manager-55b4d954d5-rz72f\" (UID: \"a4b96d6e-b281-4c96-b186-43813038ba1e\") " pod="cert-manager/cert-manager-55b4d954d5-rz72f" Sep 10 13:26:18.234444 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:18.234273 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a4b96d6e-b281-4c96-b186-43813038ba1e-bound-sa-token\") pod \"cert-manager-55b4d954d5-rz72f\" (UID: \"a4b96d6e-b281-4c96-b186-43813038ba1e\") " pod="cert-manager/cert-manager-55b4d954d5-rz72f" Sep 10 13:26:18.335700 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:18.335653 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnqpm\" (UniqueName: \"kubernetes.io/projected/a4b96d6e-b281-4c96-b186-43813038ba1e-kube-api-access-wnqpm\") pod \"cert-manager-55b4d954d5-rz72f\" (UID: \"a4b96d6e-b281-4c96-b186-43813038ba1e\") " pod="cert-manager/cert-manager-55b4d954d5-rz72f" Sep 10 13:26:18.335925 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:18.335716 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a4b96d6e-b281-4c96-b186-43813038ba1e-bound-sa-token\") pod \"cert-manager-55b4d954d5-rz72f\" (UID: \"a4b96d6e-b281-4c96-b186-43813038ba1e\") " pod="cert-manager/cert-manager-55b4d954d5-rz72f" Sep 10 13:26:18.345279 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:18.345248 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a4b96d6e-b281-4c96-b186-43813038ba1e-bound-sa-token\") pod \"cert-manager-55b4d954d5-rz72f\" (UID: \"a4b96d6e-b281-4c96-b186-43813038ba1e\") " pod="cert-manager/cert-manager-55b4d954d5-rz72f" Sep 10 13:26:18.345279 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:18.345264 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnqpm\" (UniqueName: \"kubernetes.io/projected/a4b96d6e-b281-4c96-b186-43813038ba1e-kube-api-access-wnqpm\") pod \"cert-manager-55b4d954d5-rz72f\" (UID: \"a4b96d6e-b281-4c96-b186-43813038ba1e\") " pod="cert-manager/cert-manager-55b4d954d5-rz72f" Sep 10 13:26:18.468288 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:18.468250 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-rz72f" Sep 10 13:26:18.741314 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:18.741287 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rz72f"] Sep 10 13:26:18.743898 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:26:18.743868 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda4b96d6e_b281_4c96_b186_43813038ba1e.slice/crio-489b7cd71f802e68f96c4f69925cdb1425c618958a7070b0d3393f9a3a1afcbb WatchSource:0}: Error finding container 489b7cd71f802e68f96c4f69925cdb1425c618958a7070b0d3393f9a3a1afcbb: Status 404 returned error can't find the container with id 489b7cd71f802e68f96c4f69925cdb1425c618958a7070b0d3393f9a3a1afcbb Sep 10 13:26:19.296746 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:19.296709 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-rz72f" event={"ID":"a4b96d6e-b281-4c96-b186-43813038ba1e","Type":"ContainerStarted","Data":"148e08961ba6bf5e4505ad6bc8814ca0cddf8292679d2a902a0d91f826a8c5ab"} Sep 10 13:26:19.296746 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:19.296752 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-rz72f" event={"ID":"a4b96d6e-b281-4c96-b186-43813038ba1e","Type":"ContainerStarted","Data":"489b7cd71f802e68f96c4f69925cdb1425c618958a7070b0d3393f9a3a1afcbb"} Sep 10 13:26:19.323495 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:19.323430 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-rz72f" podStartSLOduration=1.323410693 podStartE2EDuration="1.323410693s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:26:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:26:19.31594109 +0000 UTC m=+489.210658502" watchObservedRunningTime="2026-09-10 13:26:19.323410693 +0000 UTC m=+489.218128102" Sep 10 13:26:21.239187 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:21.239147 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:21.345887 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:21.345852 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:28.090488 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:28.090447 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:28.200392 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:28.200354 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:29.865490 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:29.865448 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:30.831172 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:30.831136 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56xmb"] Sep 10 13:26:30.836170 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:30.836144 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56xmb" Sep 10 13:26:30.839011 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:30.838989 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 10 13:26:30.840154 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:30.840120 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-v2fn8\"" Sep 10 13:26:30.840283 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:30.840237 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 10 13:26:30.843101 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:30.843041 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56xmb"] Sep 10 13:26:30.942311 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:30.942273 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/779921b1-165d-4387-8f3a-9ed09c37bcd8-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56xmb\" (UID: \"779921b1-165d-4387-8f3a-9ed09c37bcd8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56xmb" Sep 10 13:26:30.942710 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:30.942340 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57858\" (UniqueName: \"kubernetes.io/projected/779921b1-165d-4387-8f3a-9ed09c37bcd8-kube-api-access-57858\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56xmb\" (UID: \"779921b1-165d-4387-8f3a-9ed09c37bcd8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56xmb" Sep 10 13:26:30.942710 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:30.942377 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/779921b1-165d-4387-8f3a-9ed09c37bcd8-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56xmb\" (UID: \"779921b1-165d-4387-8f3a-9ed09c37bcd8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56xmb" Sep 10 13:26:31.043733 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:31.043679 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57858\" (UniqueName: \"kubernetes.io/projected/779921b1-165d-4387-8f3a-9ed09c37bcd8-kube-api-access-57858\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56xmb\" (UID: \"779921b1-165d-4387-8f3a-9ed09c37bcd8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56xmb" Sep 10 13:26:31.043733 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:31.043744 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/779921b1-165d-4387-8f3a-9ed09c37bcd8-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56xmb\" (UID: \"779921b1-165d-4387-8f3a-9ed09c37bcd8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56xmb" Sep 10 13:26:31.043958 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:31.043801 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/779921b1-165d-4387-8f3a-9ed09c37bcd8-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56xmb\" (UID: \"779921b1-165d-4387-8f3a-9ed09c37bcd8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56xmb" Sep 10 13:26:31.044143 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:31.044128 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/779921b1-165d-4387-8f3a-9ed09c37bcd8-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56xmb\" (UID: \"779921b1-165d-4387-8f3a-9ed09c37bcd8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56xmb" Sep 10 13:26:31.044196 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:31.044178 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/779921b1-165d-4387-8f3a-9ed09c37bcd8-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56xmb\" (UID: \"779921b1-165d-4387-8f3a-9ed09c37bcd8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56xmb" Sep 10 13:26:31.053997 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:31.053976 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57858\" (UniqueName: \"kubernetes.io/projected/779921b1-165d-4387-8f3a-9ed09c37bcd8-kube-api-access-57858\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56xmb\" (UID: \"779921b1-165d-4387-8f3a-9ed09c37bcd8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56xmb" Sep 10 13:26:31.147143 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:31.147052 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56xmb" Sep 10 13:26:31.423121 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:31.423091 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56xmb"] Sep 10 13:26:31.425516 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:26:31.425486 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod779921b1_165d_4387_8f3a_9ed09c37bcd8.slice/crio-265018834b895e4a503c05e479882d78f3a903b8eab720ed7ffe6b4e24fa983a WatchSource:0}: Error finding container 265018834b895e4a503c05e479882d78f3a903b8eab720ed7ffe6b4e24fa983a: Status 404 returned error can't find the container with id 265018834b895e4a503c05e479882d78f3a903b8eab720ed7ffe6b4e24fa983a Sep 10 13:26:32.361603 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:32.361515 2560 generic.go:356] "Generic (PLEG): container finished" podID="779921b1-165d-4387-8f3a-9ed09c37bcd8" containerID="4e6c6a29d84f869015ac98bb7331165d196741b932914e3b0a0bac563e8f4ba7" exitCode=0 Sep 10 13:26:32.361603 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:32.361594 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56xmb" event={"ID":"779921b1-165d-4387-8f3a-9ed09c37bcd8","Type":"ContainerDied","Data":"4e6c6a29d84f869015ac98bb7331165d196741b932914e3b0a0bac563e8f4ba7"} Sep 10 13:26:32.362035 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:32.361632 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56xmb" event={"ID":"779921b1-165d-4387-8f3a-9ed09c37bcd8","Type":"ContainerStarted","Data":"265018834b895e4a503c05e479882d78f3a903b8eab720ed7ffe6b4e24fa983a"} Sep 10 13:26:33.368202 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:33.368115 2560 generic.go:356] "Generic (PLEG): container finished" podID="779921b1-165d-4387-8f3a-9ed09c37bcd8" containerID="6cf1f2f33a7457157e0e32c411b2b279935bb5d369dedebf4c443cdbdfe2c774" exitCode=0 Sep 10 13:26:33.368202 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:33.368169 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56xmb" event={"ID":"779921b1-165d-4387-8f3a-9ed09c37bcd8","Type":"ContainerDied","Data":"6cf1f2f33a7457157e0e32c411b2b279935bb5d369dedebf4c443cdbdfe2c774"} Sep 10 13:26:34.374322 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:34.374278 2560 generic.go:356] "Generic (PLEG): container finished" podID="779921b1-165d-4387-8f3a-9ed09c37bcd8" containerID="670664bd2c7a6af2901fbdf86baab6d53ef626651ec820ede767965cc5a57704" exitCode=0 Sep 10 13:26:34.374737 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:34.374379 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56xmb" event={"ID":"779921b1-165d-4387-8f3a-9ed09c37bcd8","Type":"ContainerDied","Data":"670664bd2c7a6af2901fbdf86baab6d53ef626651ec820ede767965cc5a57704"} Sep 10 13:26:35.137516 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:35.137466 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:35.208250 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:35.208221 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:35.503956 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:35.503924 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56xmb" Sep 10 13:26:35.582117 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:35.582081 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-57858\" (UniqueName: \"kubernetes.io/projected/779921b1-165d-4387-8f3a-9ed09c37bcd8-kube-api-access-57858\") pod \"779921b1-165d-4387-8f3a-9ed09c37bcd8\" (UID: \"779921b1-165d-4387-8f3a-9ed09c37bcd8\") " Sep 10 13:26:35.582293 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:35.582126 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/779921b1-165d-4387-8f3a-9ed09c37bcd8-util\") pod \"779921b1-165d-4387-8f3a-9ed09c37bcd8\" (UID: \"779921b1-165d-4387-8f3a-9ed09c37bcd8\") " Sep 10 13:26:35.582293 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:35.582181 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/779921b1-165d-4387-8f3a-9ed09c37bcd8-bundle\") pod \"779921b1-165d-4387-8f3a-9ed09c37bcd8\" (UID: \"779921b1-165d-4387-8f3a-9ed09c37bcd8\") " Sep 10 13:26:35.582989 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:35.582961 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/779921b1-165d-4387-8f3a-9ed09c37bcd8-bundle" (OuterVolumeSpecName: "bundle") pod "779921b1-165d-4387-8f3a-9ed09c37bcd8" (UID: "779921b1-165d-4387-8f3a-9ed09c37bcd8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:26:35.584248 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:35.584225 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/779921b1-165d-4387-8f3a-9ed09c37bcd8-kube-api-access-57858" (OuterVolumeSpecName: "kube-api-access-57858") pod "779921b1-165d-4387-8f3a-9ed09c37bcd8" (UID: "779921b1-165d-4387-8f3a-9ed09c37bcd8"). InnerVolumeSpecName "kube-api-access-57858". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:26:35.587937 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:35.587891 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/779921b1-165d-4387-8f3a-9ed09c37bcd8-util" (OuterVolumeSpecName: "util") pod "779921b1-165d-4387-8f3a-9ed09c37bcd8" (UID: "779921b1-165d-4387-8f3a-9ed09c37bcd8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:26:35.683840 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:35.683743 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/779921b1-165d-4387-8f3a-9ed09c37bcd8-bundle\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:26:35.683840 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:35.683781 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-57858\" (UniqueName: \"kubernetes.io/projected/779921b1-165d-4387-8f3a-9ed09c37bcd8-kube-api-access-57858\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:26:35.683840 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:35.683793 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/779921b1-165d-4387-8f3a-9ed09c37bcd8-util\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:26:36.385830 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:36.385802 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56xmb" Sep 10 13:26:36.385978 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:36.385803 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56xmb" event={"ID":"779921b1-165d-4387-8f3a-9ed09c37bcd8","Type":"ContainerDied","Data":"265018834b895e4a503c05e479882d78f3a903b8eab720ed7ffe6b4e24fa983a"} Sep 10 13:26:36.385978 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:36.385934 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="265018834b895e4a503c05e479882d78f3a903b8eab720ed7ffe6b4e24fa983a" Sep 10 13:26:41.270785 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:41.270748 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tq85w"] Sep 10 13:26:41.271280 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:41.271248 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="779921b1-165d-4387-8f3a-9ed09c37bcd8" containerName="extract" Sep 10 13:26:41.271280 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:41.271268 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="779921b1-165d-4387-8f3a-9ed09c37bcd8" containerName="extract" Sep 10 13:26:41.271431 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:41.271294 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="779921b1-165d-4387-8f3a-9ed09c37bcd8" containerName="util" Sep 10 13:26:41.271431 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:41.271303 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="779921b1-165d-4387-8f3a-9ed09c37bcd8" containerName="util" Sep 10 13:26:41.271431 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:41.271312 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="779921b1-165d-4387-8f3a-9ed09c37bcd8" containerName="pull" Sep 10 13:26:41.271431 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:41.271322 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="779921b1-165d-4387-8f3a-9ed09c37bcd8" containerName="pull" Sep 10 13:26:41.271431 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:41.271426 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="779921b1-165d-4387-8f3a-9ed09c37bcd8" containerName="extract" Sep 10 13:26:41.273727 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:41.273701 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tq85w" Sep 10 13:26:41.276275 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:41.276252 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-v2fn8\"" Sep 10 13:26:41.276643 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:41.276624 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 10 13:26:41.277138 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:41.277123 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 10 13:26:41.288209 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:41.288185 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tq85w"] Sep 10 13:26:41.434417 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:41.434363 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/551786a4-acf3-4b78-9a59-6a453ab92a30-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tq85w\" (UID: \"551786a4-acf3-4b78-9a59-6a453ab92a30\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tq85w" Sep 10 13:26:41.434610 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:41.434481 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dsql\" (UniqueName: \"kubernetes.io/projected/551786a4-acf3-4b78-9a59-6a453ab92a30-kube-api-access-4dsql\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tq85w\" (UID: \"551786a4-acf3-4b78-9a59-6a453ab92a30\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tq85w" Sep 10 13:26:41.434610 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:41.434519 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/551786a4-acf3-4b78-9a59-6a453ab92a30-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tq85w\" (UID: \"551786a4-acf3-4b78-9a59-6a453ab92a30\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tq85w" Sep 10 13:26:41.535808 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:41.535713 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/551786a4-acf3-4b78-9a59-6a453ab92a30-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tq85w\" (UID: \"551786a4-acf3-4b78-9a59-6a453ab92a30\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tq85w" Sep 10 13:26:41.535808 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:41.535777 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dsql\" (UniqueName: \"kubernetes.io/projected/551786a4-acf3-4b78-9a59-6a453ab92a30-kube-api-access-4dsql\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tq85w\" (UID: \"551786a4-acf3-4b78-9a59-6a453ab92a30\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tq85w" Sep 10 13:26:41.535998 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:41.535813 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/551786a4-acf3-4b78-9a59-6a453ab92a30-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tq85w\" (UID: \"551786a4-acf3-4b78-9a59-6a453ab92a30\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tq85w" Sep 10 13:26:41.536191 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:41.536170 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/551786a4-acf3-4b78-9a59-6a453ab92a30-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tq85w\" (UID: \"551786a4-acf3-4b78-9a59-6a453ab92a30\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tq85w" Sep 10 13:26:41.536231 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:41.536182 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/551786a4-acf3-4b78-9a59-6a453ab92a30-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tq85w\" (UID: \"551786a4-acf3-4b78-9a59-6a453ab92a30\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tq85w" Sep 10 13:26:41.552826 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:41.552790 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dsql\" (UniqueName: \"kubernetes.io/projected/551786a4-acf3-4b78-9a59-6a453ab92a30-kube-api-access-4dsql\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tq85w\" (UID: \"551786a4-acf3-4b78-9a59-6a453ab92a30\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tq85w" Sep 10 13:26:41.583439 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:41.583382 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tq85w" Sep 10 13:26:41.858512 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:41.858485 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tq85w"] Sep 10 13:26:41.861329 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:26:41.861303 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod551786a4_acf3_4b78_9a59_6a453ab92a30.slice/crio-5805fe270f0dacbc04e5ff7027403917936777d7708bee46b7775d69c2d92f47 WatchSource:0}: Error finding container 5805fe270f0dacbc04e5ff7027403917936777d7708bee46b7775d69c2d92f47: Status 404 returned error can't find the container with id 5805fe270f0dacbc04e5ff7027403917936777d7708bee46b7775d69c2d92f47 Sep 10 13:26:42.416314 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:42.416279 2560 generic.go:356] "Generic (PLEG): container finished" podID="551786a4-acf3-4b78-9a59-6a453ab92a30" containerID="baa97bb553cc3b9dcd7ca6ba7cba0597902a83055fe3e7b3239e264e466fb5f1" exitCode=0 Sep 10 13:26:42.416874 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:42.416320 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tq85w" event={"ID":"551786a4-acf3-4b78-9a59-6a453ab92a30","Type":"ContainerDied","Data":"baa97bb553cc3b9dcd7ca6ba7cba0597902a83055fe3e7b3239e264e466fb5f1"} Sep 10 13:26:42.416874 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:42.416355 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tq85w" event={"ID":"551786a4-acf3-4b78-9a59-6a453ab92a30","Type":"ContainerStarted","Data":"5805fe270f0dacbc04e5ff7027403917936777d7708bee46b7775d69c2d92f47"} Sep 10 13:26:42.451671 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:42.451636 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:42.584922 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:42.584887 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:43.012972 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:43.012937 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:43.423813 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:43.423772 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tq85w" event={"ID":"551786a4-acf3-4b78-9a59-6a453ab92a30","Type":"ContainerStarted","Data":"c5e667e3ffe4db34e57601e4c81be9fcc62b7f13cb7b48cb9b56eeaaf7d0e9cc"} Sep 10 13:26:43.449042 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:43.449011 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:43.577342 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:43.577300 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84d444f55-84thz"] Sep 10 13:26:43.580094 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:43.580074 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-84thz" Sep 10 13:26:43.589085 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:43.589064 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 10 13:26:43.589207 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:43.589065 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 10 13:26:43.589351 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:43.589337 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-m6d87\"" Sep 10 13:26:43.589538 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:43.589525 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 10 13:26:43.590308 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:43.590295 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 10 13:26:43.604554 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:43.604453 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84d444f55-84thz"] Sep 10 13:26:43.609037 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:43.608997 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:43.653774 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:43.653733 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/710f018c-075a-428e-aed2-d031822357a4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84d444f55-84thz\" (UID: \"710f018c-075a-428e-aed2-d031822357a4\") " pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-84thz" Sep 10 13:26:43.653960 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:43.653873 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/710f018c-075a-428e-aed2-d031822357a4-tmp\") pod \"opendatahub-operator-controller-manager-84d444f55-84thz\" (UID: \"710f018c-075a-428e-aed2-d031822357a4\") " pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-84thz" Sep 10 13:26:43.653960 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:43.653904 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/710f018c-075a-428e-aed2-d031822357a4-manifests\") pod \"opendatahub-operator-controller-manager-84d444f55-84thz\" (UID: \"710f018c-075a-428e-aed2-d031822357a4\") " pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-84thz" Sep 10 13:26:43.653960 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:43.653935 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/710f018c-075a-428e-aed2-d031822357a4-webhook-cert\") pod \"opendatahub-operator-controller-manager-84d444f55-84thz\" (UID: \"710f018c-075a-428e-aed2-d031822357a4\") " pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-84thz" Sep 10 13:26:43.654095 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:43.653988 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t9tx\" (UniqueName: \"kubernetes.io/projected/710f018c-075a-428e-aed2-d031822357a4-kube-api-access-4t9tx\") pod \"opendatahub-operator-controller-manager-84d444f55-84thz\" (UID: \"710f018c-075a-428e-aed2-d031822357a4\") " pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-84thz" Sep 10 13:26:43.755096 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:43.755053 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/710f018c-075a-428e-aed2-d031822357a4-tmp\") pod \"opendatahub-operator-controller-manager-84d444f55-84thz\" (UID: \"710f018c-075a-428e-aed2-d031822357a4\") " pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-84thz" Sep 10 13:26:43.755096 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:43.755095 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/710f018c-075a-428e-aed2-d031822357a4-manifests\") pod \"opendatahub-operator-controller-manager-84d444f55-84thz\" (UID: \"710f018c-075a-428e-aed2-d031822357a4\") " pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-84thz" Sep 10 13:26:43.755357 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:43.755120 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/710f018c-075a-428e-aed2-d031822357a4-webhook-cert\") pod \"opendatahub-operator-controller-manager-84d444f55-84thz\" (UID: \"710f018c-075a-428e-aed2-d031822357a4\") " pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-84thz" Sep 10 13:26:43.755357 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:43.755203 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t9tx\" (UniqueName: \"kubernetes.io/projected/710f018c-075a-428e-aed2-d031822357a4-kube-api-access-4t9tx\") pod \"opendatahub-operator-controller-manager-84d444f55-84thz\" (UID: \"710f018c-075a-428e-aed2-d031822357a4\") " pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-84thz" Sep 10 13:26:43.755357 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:43.755294 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/710f018c-075a-428e-aed2-d031822357a4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84d444f55-84thz\" (UID: \"710f018c-075a-428e-aed2-d031822357a4\") " pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-84thz" Sep 10 13:26:43.755661 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:43.755628 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/710f018c-075a-428e-aed2-d031822357a4-manifests\") pod \"opendatahub-operator-controller-manager-84d444f55-84thz\" (UID: \"710f018c-075a-428e-aed2-d031822357a4\") " pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-84thz" Sep 10 13:26:43.757698 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:43.757671 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/710f018c-075a-428e-aed2-d031822357a4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84d444f55-84thz\" (UID: \"710f018c-075a-428e-aed2-d031822357a4\") " pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-84thz" Sep 10 13:26:43.757896 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:43.757875 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/710f018c-075a-428e-aed2-d031822357a4-tmp\") pod \"opendatahub-operator-controller-manager-84d444f55-84thz\" (UID: \"710f018c-075a-428e-aed2-d031822357a4\") " pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-84thz" Sep 10 13:26:43.757981 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:43.757966 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/710f018c-075a-428e-aed2-d031822357a4-webhook-cert\") pod \"opendatahub-operator-controller-manager-84d444f55-84thz\" (UID: \"710f018c-075a-428e-aed2-d031822357a4\") " pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-84thz" Sep 10 13:26:43.765651 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:43.765632 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t9tx\" (UniqueName: \"kubernetes.io/projected/710f018c-075a-428e-aed2-d031822357a4-kube-api-access-4t9tx\") pod \"opendatahub-operator-controller-manager-84d444f55-84thz\" (UID: \"710f018c-075a-428e-aed2-d031822357a4\") " pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-84thz" Sep 10 13:26:43.786880 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:43.786851 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:43.889869 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:43.889786 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-84thz" Sep 10 13:26:43.957722 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:43.957683 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:44.054043 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:44.054006 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:44.150686 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:44.150541 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:44.175310 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:44.175287 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84d444f55-84thz"] Sep 10 13:26:44.176926 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:26:44.176892 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod710f018c_075a_428e_aed2_d031822357a4.slice/crio-298f1fcbfd32f895ce7335bb4c34be49363772d3fc2b28bfd81281c67fdc07e9 WatchSource:0}: Error finding container 298f1fcbfd32f895ce7335bb4c34be49363772d3fc2b28bfd81281c67fdc07e9: Status 404 returned error can't find the container with id 298f1fcbfd32f895ce7335bb4c34be49363772d3fc2b28bfd81281c67fdc07e9 Sep 10 13:26:44.250932 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:44.250897 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:44.383926 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:44.383892 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:44.430126 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:44.430090 2560 generic.go:356] "Generic (PLEG): container finished" podID="551786a4-acf3-4b78-9a59-6a453ab92a30" containerID="c5e667e3ffe4db34e57601e4c81be9fcc62b7f13cb7b48cb9b56eeaaf7d0e9cc" exitCode=0 Sep 10 13:26:44.430554 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:44.430166 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tq85w" event={"ID":"551786a4-acf3-4b78-9a59-6a453ab92a30","Type":"ContainerDied","Data":"c5e667e3ffe4db34e57601e4c81be9fcc62b7f13cb7b48cb9b56eeaaf7d0e9cc"} Sep 10 13:26:44.431557 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:44.431537 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-84thz" event={"ID":"710f018c-075a-428e-aed2-d031822357a4","Type":"ContainerStarted","Data":"298f1fcbfd32f895ce7335bb4c34be49363772d3fc2b28bfd81281c67fdc07e9"} Sep 10 13:26:45.440949 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:45.440721 2560 generic.go:356] "Generic (PLEG): container finished" podID="551786a4-acf3-4b78-9a59-6a453ab92a30" containerID="17b3626a3530cef2f7dad9d152a9b16a059c4300d1481c37c5af78a1d573ea93" exitCode=0 Sep 10 13:26:45.440949 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:45.440850 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tq85w" event={"ID":"551786a4-acf3-4b78-9a59-6a453ab92a30","Type":"ContainerDied","Data":"17b3626a3530cef2f7dad9d152a9b16a059c4300d1481c37c5af78a1d573ea93"} Sep 10 13:26:46.203796 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:46.203756 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:46.344108 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:46.344066 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:47.154129 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:47.154097 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tq85w" Sep 10 13:26:47.285828 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:47.285785 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/551786a4-acf3-4b78-9a59-6a453ab92a30-bundle\") pod \"551786a4-acf3-4b78-9a59-6a453ab92a30\" (UID: \"551786a4-acf3-4b78-9a59-6a453ab92a30\") " Sep 10 13:26:47.285995 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:47.285885 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/551786a4-acf3-4b78-9a59-6a453ab92a30-util\") pod \"551786a4-acf3-4b78-9a59-6a453ab92a30\" (UID: \"551786a4-acf3-4b78-9a59-6a453ab92a30\") " Sep 10 13:26:47.285995 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:47.285922 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4dsql\" (UniqueName: \"kubernetes.io/projected/551786a4-acf3-4b78-9a59-6a453ab92a30-kube-api-access-4dsql\") pod \"551786a4-acf3-4b78-9a59-6a453ab92a30\" (UID: \"551786a4-acf3-4b78-9a59-6a453ab92a30\") " Sep 10 13:26:47.286519 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:47.286486 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/551786a4-acf3-4b78-9a59-6a453ab92a30-bundle" (OuterVolumeSpecName: "bundle") pod "551786a4-acf3-4b78-9a59-6a453ab92a30" (UID: "551786a4-acf3-4b78-9a59-6a453ab92a30"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:26:47.288030 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:47.288007 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/551786a4-acf3-4b78-9a59-6a453ab92a30-kube-api-access-4dsql" (OuterVolumeSpecName: "kube-api-access-4dsql") pod "551786a4-acf3-4b78-9a59-6a453ab92a30" (UID: "551786a4-acf3-4b78-9a59-6a453ab92a30"). InnerVolumeSpecName "kube-api-access-4dsql". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:26:47.290826 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:47.290803 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/551786a4-acf3-4b78-9a59-6a453ab92a30-util" (OuterVolumeSpecName: "util") pod "551786a4-acf3-4b78-9a59-6a453ab92a30" (UID: "551786a4-acf3-4b78-9a59-6a453ab92a30"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:26:47.387565 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:47.387467 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/551786a4-acf3-4b78-9a59-6a453ab92a30-util\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:26:47.387565 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:47.387503 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4dsql\" (UniqueName: \"kubernetes.io/projected/551786a4-acf3-4b78-9a59-6a453ab92a30-kube-api-access-4dsql\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:26:47.387565 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:47.387515 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/551786a4-acf3-4b78-9a59-6a453ab92a30-bundle\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:26:47.456167 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:47.456139 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tq85w" Sep 10 13:26:47.456429 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:47.456135 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tq85w" event={"ID":"551786a4-acf3-4b78-9a59-6a453ab92a30","Type":"ContainerDied","Data":"5805fe270f0dacbc04e5ff7027403917936777d7708bee46b7775d69c2d92f47"} Sep 10 13:26:47.456429 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:47.456262 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5805fe270f0dacbc04e5ff7027403917936777d7708bee46b7775d69c2d92f47" Sep 10 13:26:47.457911 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:47.457885 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-84thz" event={"ID":"710f018c-075a-428e-aed2-d031822357a4","Type":"ContainerStarted","Data":"b9be2c7ab7710e492bc09fa3d835d65f1a46bf21ec5a643728c6ef9bb9349e6b"} Sep 10 13:26:48.464674 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:48.464639 2560 generic.go:356] "Generic (PLEG): container finished" podID="710f018c-075a-428e-aed2-d031822357a4" containerID="b9be2c7ab7710e492bc09fa3d835d65f1a46bf21ec5a643728c6ef9bb9349e6b" exitCode=0 Sep 10 13:26:48.465072 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:48.464671 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-84thz" event={"ID":"710f018c-075a-428e-aed2-d031822357a4","Type":"ContainerDied","Data":"b9be2c7ab7710e492bc09fa3d835d65f1a46bf21ec5a643728c6ef9bb9349e6b"} Sep 10 13:26:49.473894 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:49.473858 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-84thz" event={"ID":"710f018c-075a-428e-aed2-d031822357a4","Type":"ContainerStarted","Data":"214e8b20c7fc810f189204989504ed7401dfac21b9c5ad00c84fcab2f50df019"} Sep 10 13:26:49.474374 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:49.473987 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-84thz" Sep 10 13:26:49.500283 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:49.500232 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-84thz" podStartSLOduration=1.959585436 podStartE2EDuration="6.500215208s" totalImagesPullingTime="3.420377485s" totalInitContainerRuntime="1.120252287s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:26:43 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:26:49.497738824 +0000 UTC m=+519.392456244" watchObservedRunningTime="2026-09-10 13:26:49.500215208 +0000 UTC m=+519.394932626" Sep 10 13:26:53.858639 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:53.858590 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:54.008150 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:54.008110 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:57.883851 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:57.883816 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:57.994898 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:57.994862 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:58.315296 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:58.315262 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:26:58.418239 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:26:58.418206 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:27:00.480764 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:00.480736 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-84d444f55-84thz" Sep 10 13:27:02.748583 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:02.748545 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6746899c78-9fm28"] Sep 10 13:27:02.749002 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:02.748853 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="551786a4-acf3-4b78-9a59-6a453ab92a30" containerName="extract" Sep 10 13:27:02.749002 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:02.748864 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="551786a4-acf3-4b78-9a59-6a453ab92a30" containerName="extract" Sep 10 13:27:02.749002 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:02.748884 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="551786a4-acf3-4b78-9a59-6a453ab92a30" containerName="util" Sep 10 13:27:02.749002 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:02.748890 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="551786a4-acf3-4b78-9a59-6a453ab92a30" containerName="util" Sep 10 13:27:02.749002 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:02.748896 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="551786a4-acf3-4b78-9a59-6a453ab92a30" containerName="pull" Sep 10 13:27:02.749002 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:02.748902 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="551786a4-acf3-4b78-9a59-6a453ab92a30" containerName="pull" Sep 10 13:27:02.749002 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:02.748951 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="551786a4-acf3-4b78-9a59-6a453ab92a30" containerName="extract" Sep 10 13:27:02.760030 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:02.760001 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6746899c78-9fm28" Sep 10 13:27:02.762776 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:02.762742 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Sep 10 13:27:02.763844 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:02.763782 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Sep 10 13:27:02.763844 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:02.763812 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Sep 10 13:27:02.763844 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:02.763831 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Sep 10 13:27:02.763844 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:02.763855 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-v4qfv\"" Sep 10 13:27:02.764168 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:02.763860 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Sep 10 13:27:02.771892 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:02.771869 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6746899c78-9fm28"] Sep 10 13:27:02.815682 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:02.815598 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkt4h\" (UniqueName: \"kubernetes.io/projected/3180edaa-1746-4080-9a8f-a0e53fb0afc6-kube-api-access-qkt4h\") pod \"lws-controller-manager-6746899c78-9fm28\" (UID: \"3180edaa-1746-4080-9a8f-a0e53fb0afc6\") " pod="openshift-lws-operator/lws-controller-manager-6746899c78-9fm28" Sep 10 13:27:02.815682 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:02.815639 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3180edaa-1746-4080-9a8f-a0e53fb0afc6-cert\") pod \"lws-controller-manager-6746899c78-9fm28\" (UID: \"3180edaa-1746-4080-9a8f-a0e53fb0afc6\") " pod="openshift-lws-operator/lws-controller-manager-6746899c78-9fm28" Sep 10 13:27:02.815682 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:02.815658 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3180edaa-1746-4080-9a8f-a0e53fb0afc6-metrics-cert\") pod \"lws-controller-manager-6746899c78-9fm28\" (UID: \"3180edaa-1746-4080-9a8f-a0e53fb0afc6\") " pod="openshift-lws-operator/lws-controller-manager-6746899c78-9fm28" Sep 10 13:27:02.815682 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:02.815677 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3180edaa-1746-4080-9a8f-a0e53fb0afc6-manager-config\") pod \"lws-controller-manager-6746899c78-9fm28\" (UID: \"3180edaa-1746-4080-9a8f-a0e53fb0afc6\") " pod="openshift-lws-operator/lws-controller-manager-6746899c78-9fm28" Sep 10 13:27:02.916568 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:02.916531 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkt4h\" (UniqueName: \"kubernetes.io/projected/3180edaa-1746-4080-9a8f-a0e53fb0afc6-kube-api-access-qkt4h\") pod \"lws-controller-manager-6746899c78-9fm28\" (UID: \"3180edaa-1746-4080-9a8f-a0e53fb0afc6\") " pod="openshift-lws-operator/lws-controller-manager-6746899c78-9fm28" Sep 10 13:27:02.916568 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:02.916570 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3180edaa-1746-4080-9a8f-a0e53fb0afc6-cert\") pod \"lws-controller-manager-6746899c78-9fm28\" (UID: \"3180edaa-1746-4080-9a8f-a0e53fb0afc6\") " pod="openshift-lws-operator/lws-controller-manager-6746899c78-9fm28" Sep 10 13:27:02.916838 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:02.916589 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3180edaa-1746-4080-9a8f-a0e53fb0afc6-metrics-cert\") pod \"lws-controller-manager-6746899c78-9fm28\" (UID: \"3180edaa-1746-4080-9a8f-a0e53fb0afc6\") " pod="openshift-lws-operator/lws-controller-manager-6746899c78-9fm28" Sep 10 13:27:02.916838 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:02.916617 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3180edaa-1746-4080-9a8f-a0e53fb0afc6-manager-config\") pod \"lws-controller-manager-6746899c78-9fm28\" (UID: \"3180edaa-1746-4080-9a8f-a0e53fb0afc6\") " pod="openshift-lws-operator/lws-controller-manager-6746899c78-9fm28" Sep 10 13:27:02.917376 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:02.917350 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3180edaa-1746-4080-9a8f-a0e53fb0afc6-manager-config\") pod \"lws-controller-manager-6746899c78-9fm28\" (UID: \"3180edaa-1746-4080-9a8f-a0e53fb0afc6\") " pod="openshift-lws-operator/lws-controller-manager-6746899c78-9fm28" Sep 10 13:27:02.919110 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:02.919082 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3180edaa-1746-4080-9a8f-a0e53fb0afc6-cert\") pod \"lws-controller-manager-6746899c78-9fm28\" (UID: \"3180edaa-1746-4080-9a8f-a0e53fb0afc6\") " pod="openshift-lws-operator/lws-controller-manager-6746899c78-9fm28" Sep 10 13:27:02.919252 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:02.919231 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3180edaa-1746-4080-9a8f-a0e53fb0afc6-metrics-cert\") pod \"lws-controller-manager-6746899c78-9fm28\" (UID: \"3180edaa-1746-4080-9a8f-a0e53fb0afc6\") " pod="openshift-lws-operator/lws-controller-manager-6746899c78-9fm28" Sep 10 13:27:02.936958 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:02.936923 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkt4h\" (UniqueName: \"kubernetes.io/projected/3180edaa-1746-4080-9a8f-a0e53fb0afc6-kube-api-access-qkt4h\") pod \"lws-controller-manager-6746899c78-9fm28\" (UID: \"3180edaa-1746-4080-9a8f-a0e53fb0afc6\") " pod="openshift-lws-operator/lws-controller-manager-6746899c78-9fm28" Sep 10 13:27:03.069956 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:03.069875 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6746899c78-9fm28" Sep 10 13:27:03.267649 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:03.267612 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:27:03.357553 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:03.357528 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6746899c78-9fm28"] Sep 10 13:27:03.360029 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:27:03.359999 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3180edaa_1746_4080_9a8f_a0e53fb0afc6.slice/crio-36d86983802afe3fc9e12a72a703aca5eca7381bcc426768b78df6c5105cd1a6 WatchSource:0}: Error finding container 36d86983802afe3fc9e12a72a703aca5eca7381bcc426768b78df6c5105cd1a6: Status 404 returned error can't find the container with id 36d86983802afe3fc9e12a72a703aca5eca7381bcc426768b78df6c5105cd1a6 Sep 10 13:27:03.422094 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:03.422052 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:27:03.540082 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:03.540047 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6746899c78-9fm28" event={"ID":"3180edaa-1746-4080-9a8f-a0e53fb0afc6","Type":"ContainerStarted","Data":"36d86983802afe3fc9e12a72a703aca5eca7381bcc426768b78df6c5105cd1a6"} Sep 10 13:27:03.617598 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:03.617522 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:27:03.830091 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:03.830055 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:27:05.222468 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:05.222441 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:27:05.379590 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:05.379531 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:27:05.551953 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:05.551919 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6746899c78-9fm28" event={"ID":"3180edaa-1746-4080-9a8f-a0e53fb0afc6","Type":"ContainerStarted","Data":"2b7d5067400af89479d8f0d15a79e39c59b63e7cf3e41feadfc8b4590fd04b65"} Sep 10 13:27:05.552143 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:05.552070 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6746899c78-9fm28" Sep 10 13:27:05.580742 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:05.580690 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6746899c78-9fm28" podStartSLOduration=1.776399623 podStartE2EDuration="3.580674341s" totalImagesPullingTime="1.804274718s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:27:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:27:05.579335179 +0000 UTC m=+535.474052600" watchObservedRunningTime="2026-09-10 13:27:05.580674341 +0000 UTC m=+535.475391758" Sep 10 13:27:06.567452 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:06.567397 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:27:11.777113 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:11.777069 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:27:11.872226 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:11.872185 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:27:12.761984 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:12.761952 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg"] Sep 10 13:27:12.766908 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:12.766889 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg" Sep 10 13:27:12.772813 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:12.772793 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 10 13:27:12.772926 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:12.772907 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-gx65q\"" Sep 10 13:27:12.782685 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:12.782660 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf02ae3-6a55-453b-b14d-d86d21aee134-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg\" (UID: \"acf02ae3-6a55-453b-b14d-d86d21aee134\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg" Sep 10 13:27:12.783070 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:12.782701 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acf02ae3-6a55-453b-b14d-d86d21aee134-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg\" (UID: \"acf02ae3-6a55-453b-b14d-d86d21aee134\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg" Sep 10 13:27:12.783070 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:12.782725 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acf02ae3-6a55-453b-b14d-d86d21aee134-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg\" (UID: \"acf02ae3-6a55-453b-b14d-d86d21aee134\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg" Sep 10 13:27:12.783070 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:12.782743 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhs6g\" (UniqueName: \"kubernetes.io/projected/acf02ae3-6a55-453b-b14d-d86d21aee134-kube-api-access-qhs6g\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg\" (UID: \"acf02ae3-6a55-453b-b14d-d86d21aee134\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg" Sep 10 13:27:12.783070 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:12.782810 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf02ae3-6a55-453b-b14d-d86d21aee134-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg\" (UID: \"acf02ae3-6a55-453b-b14d-d86d21aee134\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg" Sep 10 13:27:12.783070 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:12.782851 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acf02ae3-6a55-453b-b14d-d86d21aee134-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg\" (UID: \"acf02ae3-6a55-453b-b14d-d86d21aee134\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg" Sep 10 13:27:12.783070 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:12.782880 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acf02ae3-6a55-453b-b14d-d86d21aee134-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg\" (UID: \"acf02ae3-6a55-453b-b14d-d86d21aee134\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg" Sep 10 13:27:12.783070 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:12.782930 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acf02ae3-6a55-453b-b14d-d86d21aee134-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg\" (UID: \"acf02ae3-6a55-453b-b14d-d86d21aee134\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg" Sep 10 13:27:12.783070 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:12.782956 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acf02ae3-6a55-453b-b14d-d86d21aee134-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg\" (UID: \"acf02ae3-6a55-453b-b14d-d86d21aee134\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg" Sep 10 13:27:12.797995 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:12.797955 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg"] Sep 10 13:27:12.862884 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:12.862842 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:27:12.883913 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:12.883874 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acf02ae3-6a55-453b-b14d-d86d21aee134-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg\" (UID: \"acf02ae3-6a55-453b-b14d-d86d21aee134\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg" Sep 10 13:27:12.884084 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:12.883918 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acf02ae3-6a55-453b-b14d-d86d21aee134-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg\" (UID: \"acf02ae3-6a55-453b-b14d-d86d21aee134\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg" Sep 10 13:27:12.884084 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:12.883956 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf02ae3-6a55-453b-b14d-d86d21aee134-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg\" (UID: \"acf02ae3-6a55-453b-b14d-d86d21aee134\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg" Sep 10 13:27:12.884084 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:12.884000 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acf02ae3-6a55-453b-b14d-d86d21aee134-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg\" (UID: \"acf02ae3-6a55-453b-b14d-d86d21aee134\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg" Sep 10 13:27:12.884084 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:12.884040 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acf02ae3-6a55-453b-b14d-d86d21aee134-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg\" (UID: \"acf02ae3-6a55-453b-b14d-d86d21aee134\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg" Sep 10 13:27:12.884303 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:12.884217 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhs6g\" (UniqueName: \"kubernetes.io/projected/acf02ae3-6a55-453b-b14d-d86d21aee134-kube-api-access-qhs6g\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg\" (UID: \"acf02ae3-6a55-453b-b14d-d86d21aee134\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg" Sep 10 13:27:12.884303 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:12.884287 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf02ae3-6a55-453b-b14d-d86d21aee134-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg\" (UID: \"acf02ae3-6a55-453b-b14d-d86d21aee134\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg" Sep 10 13:27:12.884444 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:12.884327 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acf02ae3-6a55-453b-b14d-d86d21aee134-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg\" (UID: \"acf02ae3-6a55-453b-b14d-d86d21aee134\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg" Sep 10 13:27:12.884444 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:12.884338 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acf02ae3-6a55-453b-b14d-d86d21aee134-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg\" (UID: \"acf02ae3-6a55-453b-b14d-d86d21aee134\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg" Sep 10 13:27:12.884444 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:12.884352 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acf02ae3-6a55-453b-b14d-d86d21aee134-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg\" (UID: \"acf02ae3-6a55-453b-b14d-d86d21aee134\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg" Sep 10 13:27:12.884444 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:12.884430 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf02ae3-6a55-453b-b14d-d86d21aee134-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg\" (UID: \"acf02ae3-6a55-453b-b14d-d86d21aee134\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg" Sep 10 13:27:12.884712 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:12.884687 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acf02ae3-6a55-453b-b14d-d86d21aee134-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg\" (UID: \"acf02ae3-6a55-453b-b14d-d86d21aee134\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg" Sep 10 13:27:12.884947 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:12.884771 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf02ae3-6a55-453b-b14d-d86d21aee134-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg\" (UID: \"acf02ae3-6a55-453b-b14d-d86d21aee134\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg" Sep 10 13:27:12.884947 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:12.884795 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acf02ae3-6a55-453b-b14d-d86d21aee134-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg\" (UID: \"acf02ae3-6a55-453b-b14d-d86d21aee134\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg" Sep 10 13:27:12.886469 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:12.886448 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acf02ae3-6a55-453b-b14d-d86d21aee134-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg\" (UID: \"acf02ae3-6a55-453b-b14d-d86d21aee134\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg" Sep 10 13:27:12.886727 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:12.886707 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acf02ae3-6a55-453b-b14d-d86d21aee134-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg\" (UID: \"acf02ae3-6a55-453b-b14d-d86d21aee134\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg" Sep 10 13:27:12.914911 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:12.914885 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhs6g\" (UniqueName: \"kubernetes.io/projected/acf02ae3-6a55-453b-b14d-d86d21aee134-kube-api-access-qhs6g\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg\" (UID: \"acf02ae3-6a55-453b-b14d-d86d21aee134\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg" Sep 10 13:27:12.915269 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:12.915252 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acf02ae3-6a55-453b-b14d-d86d21aee134-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg\" (UID: \"acf02ae3-6a55-453b-b14d-d86d21aee134\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg" Sep 10 13:27:13.062245 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:13.062147 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:27:13.077326 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:13.077297 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg" Sep 10 13:27:13.164565 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:13.164538 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:27:13.371494 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:13.371468 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg"] Sep 10 13:27:13.372282 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:27:13.372259 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacf02ae3_6a55_453b_b14d_d86d21aee134.slice/crio-0a567debc514a97fc9cd485016696159a1b094ab04c5abebb0a0d24069f7d7fa WatchSource:0}: Error finding container 0a567debc514a97fc9cd485016696159a1b094ab04c5abebb0a0d24069f7d7fa: Status 404 returned error can't find the container with id 0a567debc514a97fc9cd485016696159a1b094ab04c5abebb0a0d24069f7d7fa Sep 10 13:27:13.590524 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:13.590490 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg" event={"ID":"acf02ae3-6a55-453b-b14d-d86d21aee134","Type":"ContainerStarted","Data":"0a567debc514a97fc9cd485016696159a1b094ab04c5abebb0a0d24069f7d7fa"} Sep 10 13:27:14.364280 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:14.364233 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-8sz42"] Sep 10 13:27:14.368679 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:14.368659 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-8sz42" Sep 10 13:27:14.372541 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:14.372500 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Sep 10 13:27:14.372673 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:14.372506 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-xl6c5\"" Sep 10 13:27:14.382811 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:14.382781 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-8sz42"] Sep 10 13:27:14.395274 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:14.395239 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62n2z\" (UniqueName: \"kubernetes.io/projected/8a279cf1-6227-43c0-98a3-9dab824bac05-kube-api-access-62n2z\") pod \"odh-model-controller-7998c67dc4-8sz42\" (UID: \"8a279cf1-6227-43c0-98a3-9dab824bac05\") " pod="opendatahub/odh-model-controller-7998c67dc4-8sz42" Sep 10 13:27:14.395419 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:14.395348 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a279cf1-6227-43c0-98a3-9dab824bac05-cert\") pod \"odh-model-controller-7998c67dc4-8sz42\" (UID: \"8a279cf1-6227-43c0-98a3-9dab824bac05\") " pod="opendatahub/odh-model-controller-7998c67dc4-8sz42" Sep 10 13:27:14.448657 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:14.448623 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:27:14.496617 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:14.496581 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a279cf1-6227-43c0-98a3-9dab824bac05-cert\") pod \"odh-model-controller-7998c67dc4-8sz42\" (UID: \"8a279cf1-6227-43c0-98a3-9dab824bac05\") " pod="opendatahub/odh-model-controller-7998c67dc4-8sz42" Sep 10 13:27:14.496809 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:14.496645 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62n2z\" (UniqueName: \"kubernetes.io/projected/8a279cf1-6227-43c0-98a3-9dab824bac05-kube-api-access-62n2z\") pod \"odh-model-controller-7998c67dc4-8sz42\" (UID: \"8a279cf1-6227-43c0-98a3-9dab824bac05\") " pod="opendatahub/odh-model-controller-7998c67dc4-8sz42" Sep 10 13:27:14.496809 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:27:14.496737 2560 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Sep 10 13:27:14.496809 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:27:14.496800 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a279cf1-6227-43c0-98a3-9dab824bac05-cert podName:8a279cf1-6227-43c0-98a3-9dab824bac05 nodeName:}" failed. No retries permitted until 2026-09-10 13:27:14.996783976 +0000 UTC m=+544.891501371 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/8a279cf1-6227-43c0-98a3-9dab824bac05-cert") pod "odh-model-controller-7998c67dc4-8sz42" (UID: "8a279cf1-6227-43c0-98a3-9dab824bac05") : secret "odh-model-controller-webhook-cert" not found Sep 10 13:27:14.511919 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:14.511893 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62n2z\" (UniqueName: \"kubernetes.io/projected/8a279cf1-6227-43c0-98a3-9dab824bac05-kube-api-access-62n2z\") pod \"odh-model-controller-7998c67dc4-8sz42\" (UID: \"8a279cf1-6227-43c0-98a3-9dab824bac05\") " pod="opendatahub/odh-model-controller-7998c67dc4-8sz42" Sep 10 13:27:14.616067 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:14.615957 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:27:14.716941 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:14.716904 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:27:14.820922 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:14.820869 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:27:15.001082 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:15.001051 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a279cf1-6227-43c0-98a3-9dab824bac05-cert\") pod \"odh-model-controller-7998c67dc4-8sz42\" (UID: \"8a279cf1-6227-43c0-98a3-9dab824bac05\") " pod="opendatahub/odh-model-controller-7998c67dc4-8sz42" Sep 10 13:27:15.003508 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:15.003484 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a279cf1-6227-43c0-98a3-9dab824bac05-cert\") pod \"odh-model-controller-7998c67dc4-8sz42\" (UID: \"8a279cf1-6227-43c0-98a3-9dab824bac05\") " pod="opendatahub/odh-model-controller-7998c67dc4-8sz42" Sep 10 13:27:15.051667 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:15.051625 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gllqt"] Sep 10 13:27:15.055088 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:15.055068 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-gllqt" Sep 10 13:27:15.057684 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:15.057664 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Sep 10 13:27:15.057814 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:15.057748 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-q7785\"" Sep 10 13:27:15.066869 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:15.066841 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gllqt"] Sep 10 13:27:15.077362 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:15.077333 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:27:15.101907 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:15.101874 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40bbd992-6076-4336-8062-20ac28f260a6-tls-certs\") pod \"model-serving-api-54d89bd5fc-gllqt\" (UID: \"40bbd992-6076-4336-8062-20ac28f260a6\") " pod="opendatahub/model-serving-api-54d89bd5fc-gllqt" Sep 10 13:27:15.102099 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:15.101921 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxbpp\" (UniqueName: \"kubernetes.io/projected/40bbd992-6076-4336-8062-20ac28f260a6-kube-api-access-wxbpp\") pod \"model-serving-api-54d89bd5fc-gllqt\" (UID: \"40bbd992-6076-4336-8062-20ac28f260a6\") " pod="opendatahub/model-serving-api-54d89bd5fc-gllqt" Sep 10 13:27:15.202854 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:15.202816 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40bbd992-6076-4336-8062-20ac28f260a6-tls-certs\") pod \"model-serving-api-54d89bd5fc-gllqt\" (UID: \"40bbd992-6076-4336-8062-20ac28f260a6\") " pod="opendatahub/model-serving-api-54d89bd5fc-gllqt" Sep 10 13:27:15.202854 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:15.202860 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxbpp\" (UniqueName: \"kubernetes.io/projected/40bbd992-6076-4336-8062-20ac28f260a6-kube-api-access-wxbpp\") pod \"model-serving-api-54d89bd5fc-gllqt\" (UID: \"40bbd992-6076-4336-8062-20ac28f260a6\") " pod="opendatahub/model-serving-api-54d89bd5fc-gllqt" Sep 10 13:27:15.203120 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:27:15.203008 2560 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Sep 10 13:27:15.203120 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:27:15.203091 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40bbd992-6076-4336-8062-20ac28f260a6-tls-certs podName:40bbd992-6076-4336-8062-20ac28f260a6 nodeName:}" failed. No retries permitted until 2026-09-10 13:27:15.703069259 +0000 UTC m=+545.597786656 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/40bbd992-6076-4336-8062-20ac28f260a6-tls-certs") pod "model-serving-api-54d89bd5fc-gllqt" (UID: "40bbd992-6076-4336-8062-20ac28f260a6") : secret "model-serving-api-tls" not found Sep 10 13:27:15.223003 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:15.222971 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxbpp\" (UniqueName: \"kubernetes.io/projected/40bbd992-6076-4336-8062-20ac28f260a6-kube-api-access-wxbpp\") pod \"model-serving-api-54d89bd5fc-gllqt\" (UID: \"40bbd992-6076-4336-8062-20ac28f260a6\") " pod="opendatahub/model-serving-api-54d89bd5fc-gllqt" Sep 10 13:27:15.279544 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:15.279456 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-8sz42" Sep 10 13:27:15.363996 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:15.363940 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:27:15.566958 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:15.566847 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-8sz42"] Sep 10 13:27:15.570738 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:27:15.570713 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a279cf1_6227_43c0_98a3_9dab824bac05.slice/crio-7555fd921ce25ee19c2d2b5cbeb3a1ef31692f9ba95b2d04467754ad34730b64 WatchSource:0}: Error finding container 7555fd921ce25ee19c2d2b5cbeb3a1ef31692f9ba95b2d04467754ad34730b64: Status 404 returned error can't find the container with id 7555fd921ce25ee19c2d2b5cbeb3a1ef31692f9ba95b2d04467754ad34730b64 Sep 10 13:27:15.602077 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:15.602045 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-8sz42" event={"ID":"8a279cf1-6227-43c0-98a3-9dab824bac05","Type":"ContainerStarted","Data":"7555fd921ce25ee19c2d2b5cbeb3a1ef31692f9ba95b2d04467754ad34730b64"} Sep 10 13:27:15.708094 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:15.708059 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40bbd992-6076-4336-8062-20ac28f260a6-tls-certs\") pod \"model-serving-api-54d89bd5fc-gllqt\" (UID: \"40bbd992-6076-4336-8062-20ac28f260a6\") " pod="opendatahub/model-serving-api-54d89bd5fc-gllqt" Sep 10 13:27:15.710470 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:15.710445 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40bbd992-6076-4336-8062-20ac28f260a6-tls-certs\") pod \"model-serving-api-54d89bd5fc-gllqt\" (UID: \"40bbd992-6076-4336-8062-20ac28f260a6\") " pod="opendatahub/model-serving-api-54d89bd5fc-gllqt" Sep 10 13:27:15.966994 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:15.966962 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-gllqt" Sep 10 13:27:16.248094 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:16.247455 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gllqt"] Sep 10 13:27:16.254695 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:27:16.254658 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40bbd992_6076_4336_8062_20ac28f260a6.slice/crio-8ac6ba2494f80c1bb055629a3e4624fcc0ad62b1a06caefb4352cdb6871e9317 WatchSource:0}: Error finding container 8ac6ba2494f80c1bb055629a3e4624fcc0ad62b1a06caefb4352cdb6871e9317: Status 404 returned error can't find the container with id 8ac6ba2494f80c1bb055629a3e4624fcc0ad62b1a06caefb4352cdb6871e9317 Sep 10 13:27:16.567918 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:16.567223 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6746899c78-9fm28" Sep 10 13:27:16.614184 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:16.614098 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-gllqt" event={"ID":"40bbd992-6076-4336-8062-20ac28f260a6","Type":"ContainerStarted","Data":"8ac6ba2494f80c1bb055629a3e4624fcc0ad62b1a06caefb4352cdb6871e9317"} Sep 10 13:27:16.973377 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:16.973297 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:27:17.056028 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:17.055987 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:27:20.735241 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:20.735200 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:27:20.735612 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:20.735279 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:27:20.735612 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:20.735314 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:27:21.506703 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:21.506665 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-5kfd2"] Sep 10 13:27:21.510410 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:21.510379 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5kfd2" Sep 10 13:27:21.513237 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:21.513194 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Sep 10 13:27:21.513439 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:21.513380 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Sep 10 13:27:21.513528 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:21.513513 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Sep 10 13:27:21.514209 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:21.514193 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-4tc5x\"" Sep 10 13:27:21.538425 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:21.537797 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-5kfd2"] Sep 10 13:27:21.560545 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:21.560498 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/04fbc9f1-fede-4e96-b94d-9dc07970c8a7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-5kfd2\" (UID: \"04fbc9f1-fede-4e96-b94d-9dc07970c8a7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5kfd2" Sep 10 13:27:21.560545 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:21.560547 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hd8g\" (UniqueName: \"kubernetes.io/projected/04fbc9f1-fede-4e96-b94d-9dc07970c8a7-kube-api-access-9hd8g\") pod \"istiod-openshift-gateway-5677ccb49b-5kfd2\" (UID: \"04fbc9f1-fede-4e96-b94d-9dc07970c8a7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5kfd2" Sep 10 13:27:21.560792 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:21.560652 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/04fbc9f1-fede-4e96-b94d-9dc07970c8a7-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-5kfd2\" (UID: \"04fbc9f1-fede-4e96-b94d-9dc07970c8a7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5kfd2" Sep 10 13:27:21.560792 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:21.560688 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/04fbc9f1-fede-4e96-b94d-9dc07970c8a7-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-5kfd2\" (UID: \"04fbc9f1-fede-4e96-b94d-9dc07970c8a7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5kfd2" Sep 10 13:27:21.560792 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:21.560714 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/04fbc9f1-fede-4e96-b94d-9dc07970c8a7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-5kfd2\" (UID: \"04fbc9f1-fede-4e96-b94d-9dc07970c8a7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5kfd2" Sep 10 13:27:21.560792 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:21.560770 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/04fbc9f1-fede-4e96-b94d-9dc07970c8a7-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-5kfd2\" (UID: \"04fbc9f1-fede-4e96-b94d-9dc07970c8a7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5kfd2" Sep 10 13:27:21.561018 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:21.560804 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04fbc9f1-fede-4e96-b94d-9dc07970c8a7-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-5kfd2\" (UID: \"04fbc9f1-fede-4e96-b94d-9dc07970c8a7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5kfd2" Sep 10 13:27:21.643567 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:21.643526 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg" event={"ID":"acf02ae3-6a55-453b-b14d-d86d21aee134","Type":"ContainerStarted","Data":"ce2e2e51e51ff5ed5b53098d1bd1d8b594966510888279d042f9001bc92b0e83"} Sep 10 13:27:21.645115 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:21.645084 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-gllqt" event={"ID":"40bbd992-6076-4336-8062-20ac28f260a6","Type":"ContainerStarted","Data":"ed56da6fb94d6daaa8737583e4b7a1976cbc2d68082347b4d634067bdede5b82"} Sep 10 13:27:21.645263 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:21.645184 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-gllqt" Sep 10 13:27:21.646647 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:21.646625 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-8sz42" event={"ID":"8a279cf1-6227-43c0-98a3-9dab824bac05","Type":"ContainerStarted","Data":"28b73a4d334b215929555150c9b5ea476ea946dc3cf05f974589a7c41ddc18ae"} Sep 10 13:27:21.646779 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:21.646681 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-8sz42" Sep 10 13:27:21.661855 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:21.661823 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/04fbc9f1-fede-4e96-b94d-9dc07970c8a7-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-5kfd2\" (UID: \"04fbc9f1-fede-4e96-b94d-9dc07970c8a7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5kfd2" Sep 10 13:27:21.662058 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:21.662033 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/04fbc9f1-fede-4e96-b94d-9dc07970c8a7-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-5kfd2\" (UID: \"04fbc9f1-fede-4e96-b94d-9dc07970c8a7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5kfd2" Sep 10 13:27:21.662156 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:21.662099 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/04fbc9f1-fede-4e96-b94d-9dc07970c8a7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-5kfd2\" (UID: \"04fbc9f1-fede-4e96-b94d-9dc07970c8a7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5kfd2" Sep 10 13:27:21.662213 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:21.662196 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/04fbc9f1-fede-4e96-b94d-9dc07970c8a7-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-5kfd2\" (UID: \"04fbc9f1-fede-4e96-b94d-9dc07970c8a7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5kfd2" Sep 10 13:27:21.662267 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:21.662235 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04fbc9f1-fede-4e96-b94d-9dc07970c8a7-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-5kfd2\" (UID: \"04fbc9f1-fede-4e96-b94d-9dc07970c8a7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5kfd2" Sep 10 13:27:21.662319 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:21.662301 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/04fbc9f1-fede-4e96-b94d-9dc07970c8a7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-5kfd2\" (UID: \"04fbc9f1-fede-4e96-b94d-9dc07970c8a7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5kfd2" Sep 10 13:27:21.662390 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:21.662332 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hd8g\" (UniqueName: \"kubernetes.io/projected/04fbc9f1-fede-4e96-b94d-9dc07970c8a7-kube-api-access-9hd8g\") pod \"istiod-openshift-gateway-5677ccb49b-5kfd2\" (UID: \"04fbc9f1-fede-4e96-b94d-9dc07970c8a7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5kfd2" Sep 10 13:27:21.663356 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:21.663319 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/04fbc9f1-fede-4e96-b94d-9dc07970c8a7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-5kfd2\" (UID: \"04fbc9f1-fede-4e96-b94d-9dc07970c8a7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5kfd2" Sep 10 13:27:21.664740 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:21.664719 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/04fbc9f1-fede-4e96-b94d-9dc07970c8a7-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-5kfd2\" (UID: \"04fbc9f1-fede-4e96-b94d-9dc07970c8a7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5kfd2" Sep 10 13:27:21.664849 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:21.664737 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/04fbc9f1-fede-4e96-b94d-9dc07970c8a7-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-5kfd2\" (UID: \"04fbc9f1-fede-4e96-b94d-9dc07970c8a7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5kfd2" Sep 10 13:27:21.664849 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:21.664801 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/04fbc9f1-fede-4e96-b94d-9dc07970c8a7-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-5kfd2\" (UID: \"04fbc9f1-fede-4e96-b94d-9dc07970c8a7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5kfd2" Sep 10 13:27:21.664945 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:21.664869 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/04fbc9f1-fede-4e96-b94d-9dc07970c8a7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-5kfd2\" (UID: \"04fbc9f1-fede-4e96-b94d-9dc07970c8a7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5kfd2" Sep 10 13:27:21.667140 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:21.667093 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg" podStartSLOduration=2.306380918 podStartE2EDuration="9.667077563s" totalImagesPullingTime="7.360696645s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:27:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:27:21.665182402 +0000 UTC m=+551.559899821" watchObservedRunningTime="2026-09-10 13:27:21.667077563 +0000 UTC m=+551.561794983" Sep 10 13:27:21.674588 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:21.674569 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04fbc9f1-fede-4e96-b94d-9dc07970c8a7-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-5kfd2\" (UID: \"04fbc9f1-fede-4e96-b94d-9dc07970c8a7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5kfd2" Sep 10 13:27:21.675052 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:21.674973 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hd8g\" (UniqueName: \"kubernetes.io/projected/04fbc9f1-fede-4e96-b94d-9dc07970c8a7-kube-api-access-9hd8g\") pod \"istiod-openshift-gateway-5677ccb49b-5kfd2\" (UID: \"04fbc9f1-fede-4e96-b94d-9dc07970c8a7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5kfd2" Sep 10 13:27:21.686159 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:21.686115 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-gllqt" podStartSLOduration=2.165835899 podStartE2EDuration="6.686089964s" totalImagesPullingTime="4.520254065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:27:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:27:21.685201711 +0000 UTC m=+551.579919175" watchObservedRunningTime="2026-09-10 13:27:21.686089964 +0000 UTC m=+551.580807376" Sep 10 13:27:21.709097 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:21.709037 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-8sz42" podStartSLOduration=2.547743006 podStartE2EDuration="7.709012984s" totalImagesPullingTime="5.161269978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:27:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:27:21.708737839 +0000 UTC m=+551.603455259" watchObservedRunningTime="2026-09-10 13:27:21.709012984 +0000 UTC m=+551.603730403" Sep 10 13:27:21.819971 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:21.819885 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5kfd2" Sep 10 13:27:22.077778 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:22.077700 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg" Sep 10 13:27:22.082434 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:22.082393 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg" Sep 10 13:27:22.134905 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:22.134866 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-5kfd2"] Sep 10 13:27:22.137816 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:27:22.137775 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04fbc9f1_fede_4e96_b94d_9dc07970c8a7.slice/crio-5ea417aa98b52c937ab3ce17d022988b9e09c73ef51cac9615e2b005db81b9b3 WatchSource:0}: Error finding container 5ea417aa98b52c937ab3ce17d022988b9e09c73ef51cac9615e2b005db81b9b3: Status 404 returned error can't find the container with id 5ea417aa98b52c937ab3ce17d022988b9e09c73ef51cac9615e2b005db81b9b3 Sep 10 13:27:22.653131 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:22.653083 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5kfd2" event={"ID":"04fbc9f1-fede-4e96-b94d-9dc07970c8a7","Type":"ContainerStarted","Data":"5ea417aa98b52c937ab3ce17d022988b9e09c73ef51cac9615e2b005db81b9b3"} Sep 10 13:27:22.653334 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:22.653280 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg" Sep 10 13:27:22.654293 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:22.654269 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg" Sep 10 13:27:25.558814 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:25.558781 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:27:25.832936 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:25.832829 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:27:25.966989 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:25.966952 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:27:26.135723 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:26.135678 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:27:26.135816 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:26.135750 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:27:26.677485 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:26.677438 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5kfd2" event={"ID":"04fbc9f1-fede-4e96-b94d-9dc07970c8a7","Type":"ContainerStarted","Data":"78b25533ff22e6b58b77fbc4bda6ee654bf8c37b42fe70280b1bdc5b068366cd"} Sep 10 13:27:26.677485 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:26.677515 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5kfd2" Sep 10 13:27:26.754209 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:26.754124 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5kfd2" podStartSLOduration=1.758609085 podStartE2EDuration="5.754098752s" totalImagesPullingTime="3.995489667s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:27:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:27:26.751270552 +0000 UTC m=+556.645987972" watchObservedRunningTime="2026-09-10 13:27:26.754098752 +0000 UTC m=+556.648816170" Sep 10 13:27:27.033535 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:27.033444 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:27:27.137930 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:27.137897 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:27:27.694416 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:27.694368 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-5kfd2" Sep 10 13:27:28.528789 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:28.528745 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:27:32.655650 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:32.655614 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-8sz42" Sep 10 13:27:32.657616 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:32.657596 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-gllqt" Sep 10 13:27:53.036180 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:53.036140 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:27:53.380222 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:53.380125 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:27:53.469189 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:27:53.469155 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:28:10.470889 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:10.470862 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gtqlh_52fceedd-00f2-4943-ae47-dec51216d8e4/console-operator/2.log" Sep 10 13:28:10.471378 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:10.470860 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gtqlh_52fceedd-00f2-4943-ae47-dec51216d8e4/console-operator/2.log" Sep 10 13:28:15.396136 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:15.396090 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:28:15.555505 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:15.555471 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:28:19.990872 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:19.990832 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atxlxt"] Sep 10 13:28:19.993090 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:19.993073 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atxlxt" Sep 10 13:28:19.996488 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:19.996465 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 10 13:28:19.996632 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:19.996614 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-v2fn8\"" Sep 10 13:28:19.997369 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:19.997353 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 10 13:28:20.008946 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.008915 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atxlxt"] Sep 10 13:28:20.066301 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.066259 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d707d982-f9bb-4285-97d3-73c035c4f057-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atxlxt\" (UID: \"d707d982-f9bb-4285-97d3-73c035c4f057\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atxlxt" Sep 10 13:28:20.066512 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.066323 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d707d982-f9bb-4285-97d3-73c035c4f057-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atxlxt\" (UID: \"d707d982-f9bb-4285-97d3-73c035c4f057\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atxlxt" Sep 10 13:28:20.066512 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.066383 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhjrg\" (UniqueName: \"kubernetes.io/projected/d707d982-f9bb-4285-97d3-73c035c4f057-kube-api-access-hhjrg\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atxlxt\" (UID: \"d707d982-f9bb-4285-97d3-73c035c4f057\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atxlxt" Sep 10 13:28:20.091725 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.091684 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdzs8"] Sep 10 13:28:20.094202 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.094182 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdzs8" Sep 10 13:28:20.108377 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.108343 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdzs8"] Sep 10 13:28:20.167707 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.167667 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d707d982-f9bb-4285-97d3-73c035c4f057-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atxlxt\" (UID: \"d707d982-f9bb-4285-97d3-73c035c4f057\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atxlxt" Sep 10 13:28:20.167888 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.167723 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d362a34-f95f-41ad-b059-086c60b9c775-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdzs8\" (UID: \"7d362a34-f95f-41ad-b059-086c60b9c775\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdzs8" Sep 10 13:28:20.167888 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.167749 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhjrg\" (UniqueName: \"kubernetes.io/projected/d707d982-f9bb-4285-97d3-73c035c4f057-kube-api-access-hhjrg\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atxlxt\" (UID: \"d707d982-f9bb-4285-97d3-73c035c4f057\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atxlxt" Sep 10 13:28:20.167888 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.167780 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d362a34-f95f-41ad-b059-086c60b9c775-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdzs8\" (UID: \"7d362a34-f95f-41ad-b059-086c60b9c775\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdzs8" Sep 10 13:28:20.167888 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.167805 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d707d982-f9bb-4285-97d3-73c035c4f057-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atxlxt\" (UID: \"d707d982-f9bb-4285-97d3-73c035c4f057\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atxlxt" Sep 10 13:28:20.167888 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.167823 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjkbm\" (UniqueName: \"kubernetes.io/projected/7d362a34-f95f-41ad-b059-086c60b9c775-kube-api-access-jjkbm\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdzs8\" (UID: \"7d362a34-f95f-41ad-b059-086c60b9c775\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdzs8" Sep 10 13:28:20.168074 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.168043 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d707d982-f9bb-4285-97d3-73c035c4f057-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atxlxt\" (UID: \"d707d982-f9bb-4285-97d3-73c035c4f057\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atxlxt" Sep 10 13:28:20.168134 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.168115 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d707d982-f9bb-4285-97d3-73c035c4f057-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atxlxt\" (UID: \"d707d982-f9bb-4285-97d3-73c035c4f057\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atxlxt" Sep 10 13:28:20.180326 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.180287 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhjrg\" (UniqueName: \"kubernetes.io/projected/d707d982-f9bb-4285-97d3-73c035c4f057-kube-api-access-hhjrg\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atxlxt\" (UID: \"d707d982-f9bb-4285-97d3-73c035c4f057\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atxlxt" Sep 10 13:28:20.192813 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.192779 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zvhf2"] Sep 10 13:28:20.195390 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.195372 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zvhf2" Sep 10 13:28:20.208578 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.208547 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zvhf2"] Sep 10 13:28:20.268788 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.268697 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d362a34-f95f-41ad-b059-086c60b9c775-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdzs8\" (UID: \"7d362a34-f95f-41ad-b059-086c60b9c775\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdzs8" Sep 10 13:28:20.268788 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.268740 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/57831dfb-1fc9-4b0f-8c92-a9247c8a51ab-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zvhf2\" (UID: \"57831dfb-1fc9-4b0f-8c92-a9247c8a51ab\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zvhf2" Sep 10 13:28:20.268788 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.268774 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjkbm\" (UniqueName: \"kubernetes.io/projected/7d362a34-f95f-41ad-b059-086c60b9c775-kube-api-access-jjkbm\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdzs8\" (UID: \"7d362a34-f95f-41ad-b059-086c60b9c775\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdzs8" Sep 10 13:28:20.269018 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.268805 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mr52g\" (UniqueName: \"kubernetes.io/projected/57831dfb-1fc9-4b0f-8c92-a9247c8a51ab-kube-api-access-mr52g\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zvhf2\" (UID: \"57831dfb-1fc9-4b0f-8c92-a9247c8a51ab\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zvhf2" Sep 10 13:28:20.269018 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.268898 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d362a34-f95f-41ad-b059-086c60b9c775-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdzs8\" (UID: \"7d362a34-f95f-41ad-b059-086c60b9c775\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdzs8" Sep 10 13:28:20.269018 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.268946 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/57831dfb-1fc9-4b0f-8c92-a9247c8a51ab-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zvhf2\" (UID: \"57831dfb-1fc9-4b0f-8c92-a9247c8a51ab\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zvhf2" Sep 10 13:28:20.269131 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.269094 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d362a34-f95f-41ad-b059-086c60b9c775-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdzs8\" (UID: \"7d362a34-f95f-41ad-b059-086c60b9c775\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdzs8" Sep 10 13:28:20.269240 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.269224 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d362a34-f95f-41ad-b059-086c60b9c775-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdzs8\" (UID: \"7d362a34-f95f-41ad-b059-086c60b9c775\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdzs8" Sep 10 13:28:20.282175 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.282139 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjkbm\" (UniqueName: \"kubernetes.io/projected/7d362a34-f95f-41ad-b059-086c60b9c775-kube-api-access-jjkbm\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdzs8\" (UID: \"7d362a34-f95f-41ad-b059-086c60b9c775\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdzs8" Sep 10 13:28:20.296425 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.296379 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5887h"] Sep 10 13:28:20.298865 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.298847 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5887h" Sep 10 13:28:20.302247 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.302224 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atxlxt" Sep 10 13:28:20.313522 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.313496 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5887h"] Sep 10 13:28:20.370410 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.370364 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b7e242b-86ea-463d-9ac7-e449db63a3a2-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5887h\" (UID: \"0b7e242b-86ea-463d-9ac7-e449db63a3a2\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5887h" Sep 10 13:28:20.370556 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.370451 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mr52g\" (UniqueName: \"kubernetes.io/projected/57831dfb-1fc9-4b0f-8c92-a9247c8a51ab-kube-api-access-mr52g\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zvhf2\" (UID: \"57831dfb-1fc9-4b0f-8c92-a9247c8a51ab\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zvhf2" Sep 10 13:28:20.370556 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.370514 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/57831dfb-1fc9-4b0f-8c92-a9247c8a51ab-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zvhf2\" (UID: \"57831dfb-1fc9-4b0f-8c92-a9247c8a51ab\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zvhf2" Sep 10 13:28:20.370634 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.370565 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/57831dfb-1fc9-4b0f-8c92-a9247c8a51ab-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zvhf2\" (UID: \"57831dfb-1fc9-4b0f-8c92-a9247c8a51ab\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zvhf2" Sep 10 13:28:20.370634 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.370592 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b7e242b-86ea-463d-9ac7-e449db63a3a2-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5887h\" (UID: \"0b7e242b-86ea-463d-9ac7-e449db63a3a2\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5887h" Sep 10 13:28:20.370634 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.370626 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hq5qz\" (UniqueName: \"kubernetes.io/projected/0b7e242b-86ea-463d-9ac7-e449db63a3a2-kube-api-access-hq5qz\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5887h\" (UID: \"0b7e242b-86ea-463d-9ac7-e449db63a3a2\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5887h" Sep 10 13:28:20.371231 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.371210 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/57831dfb-1fc9-4b0f-8c92-a9247c8a51ab-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zvhf2\" (UID: \"57831dfb-1fc9-4b0f-8c92-a9247c8a51ab\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zvhf2" Sep 10 13:28:20.371274 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.371244 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/57831dfb-1fc9-4b0f-8c92-a9247c8a51ab-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zvhf2\" (UID: \"57831dfb-1fc9-4b0f-8c92-a9247c8a51ab\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zvhf2" Sep 10 13:28:20.381379 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.381350 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mr52g\" (UniqueName: \"kubernetes.io/projected/57831dfb-1fc9-4b0f-8c92-a9247c8a51ab-kube-api-access-mr52g\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zvhf2\" (UID: \"57831dfb-1fc9-4b0f-8c92-a9247c8a51ab\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zvhf2" Sep 10 13:28:20.403128 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.403090 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdzs8" Sep 10 13:28:20.471980 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.471948 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b7e242b-86ea-463d-9ac7-e449db63a3a2-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5887h\" (UID: \"0b7e242b-86ea-463d-9ac7-e449db63a3a2\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5887h" Sep 10 13:28:20.472147 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.471993 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hq5qz\" (UniqueName: \"kubernetes.io/projected/0b7e242b-86ea-463d-9ac7-e449db63a3a2-kube-api-access-hq5qz\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5887h\" (UID: \"0b7e242b-86ea-463d-9ac7-e449db63a3a2\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5887h" Sep 10 13:28:20.472147 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.472017 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b7e242b-86ea-463d-9ac7-e449db63a3a2-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5887h\" (UID: \"0b7e242b-86ea-463d-9ac7-e449db63a3a2\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5887h" Sep 10 13:28:20.472371 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.472353 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b7e242b-86ea-463d-9ac7-e449db63a3a2-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5887h\" (UID: \"0b7e242b-86ea-463d-9ac7-e449db63a3a2\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5887h" Sep 10 13:28:20.472465 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.472442 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b7e242b-86ea-463d-9ac7-e449db63a3a2-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5887h\" (UID: \"0b7e242b-86ea-463d-9ac7-e449db63a3a2\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5887h" Sep 10 13:28:20.482967 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.482936 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hq5qz\" (UniqueName: \"kubernetes.io/projected/0b7e242b-86ea-463d-9ac7-e449db63a3a2-kube-api-access-hq5qz\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5887h\" (UID: \"0b7e242b-86ea-463d-9ac7-e449db63a3a2\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5887h" Sep 10 13:28:20.504557 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.504510 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zvhf2" Sep 10 13:28:20.593561 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.593529 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atxlxt"] Sep 10 13:28:20.596766 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:28:20.596742 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd707d982_f9bb_4285_97d3_73c035c4f057.slice/crio-ba602446824dfaba5e79e25ad0fb6eafbe2376eed7190e2373d50f45a0939a36 WatchSource:0}: Error finding container ba602446824dfaba5e79e25ad0fb6eafbe2376eed7190e2373d50f45a0939a36: Status 404 returned error can't find the container with id ba602446824dfaba5e79e25ad0fb6eafbe2376eed7190e2373d50f45a0939a36 Sep 10 13:28:20.608104 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.608081 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5887h" Sep 10 13:28:20.685939 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.685906 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdzs8"] Sep 10 13:28:20.688270 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:28:20.688223 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d362a34_f95f_41ad_b059_086c60b9c775.slice/crio-a024422c6b8e2321e625a5dc494421685dfe44b27ce0652e973d88aa87aad6f3 WatchSource:0}: Error finding container a024422c6b8e2321e625a5dc494421685dfe44b27ce0652e973d88aa87aad6f3: Status 404 returned error can't find the container with id a024422c6b8e2321e625a5dc494421685dfe44b27ce0652e973d88aa87aad6f3 Sep 10 13:28:20.784389 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.784357 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zvhf2"] Sep 10 13:28:20.797669 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:28:20.797636 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57831dfb_1fc9_4b0f_8c92_a9247c8a51ab.slice/crio-11882673c7720ac31216012f23b6af7ee9dc39e62295eb9feb1d0d026d5fb9d2 WatchSource:0}: Error finding container 11882673c7720ac31216012f23b6af7ee9dc39e62295eb9feb1d0d026d5fb9d2: Status 404 returned error can't find the container with id 11882673c7720ac31216012f23b6af7ee9dc39e62295eb9feb1d0d026d5fb9d2 Sep 10 13:28:20.888975 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.888948 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5887h"] Sep 10 13:28:20.891119 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:28:20.891078 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b7e242b_86ea_463d_9ac7_e449db63a3a2.slice/crio-e359dcac19cd7ddccee9a763a2f5ea247a3b611ef1804ab43e33ce118576b154 WatchSource:0}: Error finding container e359dcac19cd7ddccee9a763a2f5ea247a3b611ef1804ab43e33ce118576b154: Status 404 returned error can't find the container with id e359dcac19cd7ddccee9a763a2f5ea247a3b611ef1804ab43e33ce118576b154 Sep 10 13:28:20.960792 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.960756 2560 generic.go:356] "Generic (PLEG): container finished" podID="57831dfb-1fc9-4b0f-8c92-a9247c8a51ab" containerID="1a763619df0657ef25b8c3a1fa3b37bbccb22e9f9b87a6988ba60bf65c679936" exitCode=0 Sep 10 13:28:20.960975 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.960844 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zvhf2" event={"ID":"57831dfb-1fc9-4b0f-8c92-a9247c8a51ab","Type":"ContainerDied","Data":"1a763619df0657ef25b8c3a1fa3b37bbccb22e9f9b87a6988ba60bf65c679936"} Sep 10 13:28:20.960975 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.960897 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zvhf2" event={"ID":"57831dfb-1fc9-4b0f-8c92-a9247c8a51ab","Type":"ContainerStarted","Data":"11882673c7720ac31216012f23b6af7ee9dc39e62295eb9feb1d0d026d5fb9d2"} Sep 10 13:28:20.962498 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.962478 2560 generic.go:356] "Generic (PLEG): container finished" podID="d707d982-f9bb-4285-97d3-73c035c4f057" containerID="a429c24a0beedc1ff5130417b02da0fb5524737ee315f8728155052632db16fc" exitCode=0 Sep 10 13:28:20.962589 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.962570 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atxlxt" event={"ID":"d707d982-f9bb-4285-97d3-73c035c4f057","Type":"ContainerDied","Data":"a429c24a0beedc1ff5130417b02da0fb5524737ee315f8728155052632db16fc"} Sep 10 13:28:20.962659 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.962616 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atxlxt" event={"ID":"d707d982-f9bb-4285-97d3-73c035c4f057","Type":"ContainerStarted","Data":"ba602446824dfaba5e79e25ad0fb6eafbe2376eed7190e2373d50f45a0939a36"} Sep 10 13:28:20.964120 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.964103 2560 generic.go:356] "Generic (PLEG): container finished" podID="7d362a34-f95f-41ad-b059-086c60b9c775" containerID="bcb6ac65a7646f0c2526c4e78a4bbe40ecacfdaa1852344ab5940783c5f79511" exitCode=0 Sep 10 13:28:20.964207 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.964186 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdzs8" event={"ID":"7d362a34-f95f-41ad-b059-086c60b9c775","Type":"ContainerDied","Data":"bcb6ac65a7646f0c2526c4e78a4bbe40ecacfdaa1852344ab5940783c5f79511"} Sep 10 13:28:20.964268 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.964231 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdzs8" event={"ID":"7d362a34-f95f-41ad-b059-086c60b9c775","Type":"ContainerStarted","Data":"a024422c6b8e2321e625a5dc494421685dfe44b27ce0652e973d88aa87aad6f3"} Sep 10 13:28:20.965510 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:20.965457 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5887h" event={"ID":"0b7e242b-86ea-463d-9ac7-e449db63a3a2","Type":"ContainerStarted","Data":"e359dcac19cd7ddccee9a763a2f5ea247a3b611ef1804ab43e33ce118576b154"} Sep 10 13:28:21.976099 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:21.976072 2560 generic.go:356] "Generic (PLEG): container finished" podID="0b7e242b-86ea-463d-9ac7-e449db63a3a2" containerID="146d192dd7c835aaa07d48c2242558bab9d8f19d7c2510196cfd0018f78aa9a2" exitCode=0 Sep 10 13:28:21.976456 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:21.976151 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5887h" event={"ID":"0b7e242b-86ea-463d-9ac7-e449db63a3a2","Type":"ContainerDied","Data":"146d192dd7c835aaa07d48c2242558bab9d8f19d7c2510196cfd0018f78aa9a2"} Sep 10 13:28:21.978107 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:21.977982 2560 generic.go:356] "Generic (PLEG): container finished" podID="57831dfb-1fc9-4b0f-8c92-a9247c8a51ab" containerID="83920e99cf4cc26388397539435bb1803e3fd572d62bb2c05662787043355a0f" exitCode=0 Sep 10 13:28:21.978107 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:21.978058 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zvhf2" event={"ID":"57831dfb-1fc9-4b0f-8c92-a9247c8a51ab","Type":"ContainerDied","Data":"83920e99cf4cc26388397539435bb1803e3fd572d62bb2c05662787043355a0f"} Sep 10 13:28:21.980215 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:21.980191 2560 generic.go:356] "Generic (PLEG): container finished" podID="7d362a34-f95f-41ad-b059-086c60b9c775" containerID="73765dd3bb1faffcfdd3f3b29abb1736ebda89b6a7921ab8b2c6e7936284d662" exitCode=0 Sep 10 13:28:21.980315 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:21.980243 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdzs8" event={"ID":"7d362a34-f95f-41ad-b059-086c60b9c775","Type":"ContainerDied","Data":"73765dd3bb1faffcfdd3f3b29abb1736ebda89b6a7921ab8b2c6e7936284d662"} Sep 10 13:28:22.987636 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:22.987549 2560 generic.go:356] "Generic (PLEG): container finished" podID="0b7e242b-86ea-463d-9ac7-e449db63a3a2" containerID="89883c4e78d00890d1ed489a6feba1a245844f5768d4dd2369e7c5a568aef9a5" exitCode=0 Sep 10 13:28:22.988059 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:22.987640 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5887h" event={"ID":"0b7e242b-86ea-463d-9ac7-e449db63a3a2","Type":"ContainerDied","Data":"89883c4e78d00890d1ed489a6feba1a245844f5768d4dd2369e7c5a568aef9a5"} Sep 10 13:28:22.989932 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:22.989911 2560 generic.go:356] "Generic (PLEG): container finished" podID="57831dfb-1fc9-4b0f-8c92-a9247c8a51ab" containerID="c9e527f0366382f89c78357b918992ab2699efb3fd6d61fa05dacdd10882ab18" exitCode=0 Sep 10 13:28:22.990009 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:22.989975 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zvhf2" event={"ID":"57831dfb-1fc9-4b0f-8c92-a9247c8a51ab","Type":"ContainerDied","Data":"c9e527f0366382f89c78357b918992ab2699efb3fd6d61fa05dacdd10882ab18"} Sep 10 13:28:22.992006 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:22.991978 2560 generic.go:356] "Generic (PLEG): container finished" podID="d707d982-f9bb-4285-97d3-73c035c4f057" containerID="10fc840e72a294c937cd633ecbb08981b447ef72d2fb22404df7b1c6750370ac" exitCode=0 Sep 10 13:28:22.992117 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:22.992063 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atxlxt" event={"ID":"d707d982-f9bb-4285-97d3-73c035c4f057","Type":"ContainerDied","Data":"10fc840e72a294c937cd633ecbb08981b447ef72d2fb22404df7b1c6750370ac"} Sep 10 13:28:22.994303 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:22.994284 2560 generic.go:356] "Generic (PLEG): container finished" podID="7d362a34-f95f-41ad-b059-086c60b9c775" containerID="6d2c43fc6361be345080f77b109e5aeede2633c55f46368281b55209356a8da4" exitCode=0 Sep 10 13:28:22.994482 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:22.994339 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdzs8" event={"ID":"7d362a34-f95f-41ad-b059-086c60b9c775","Type":"ContainerDied","Data":"6d2c43fc6361be345080f77b109e5aeede2633c55f46368281b55209356a8da4"} Sep 10 13:28:24.002291 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:24.002250 2560 generic.go:356] "Generic (PLEG): container finished" podID="0b7e242b-86ea-463d-9ac7-e449db63a3a2" containerID="7f66b24db37698051f7573488576c7f11b00543ce58abe2adae062cc90d46e7a" exitCode=0 Sep 10 13:28:24.002762 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:24.002342 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5887h" event={"ID":"0b7e242b-86ea-463d-9ac7-e449db63a3a2","Type":"ContainerDied","Data":"7f66b24db37698051f7573488576c7f11b00543ce58abe2adae062cc90d46e7a"} Sep 10 13:28:24.004534 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:24.004510 2560 generic.go:356] "Generic (PLEG): container finished" podID="d707d982-f9bb-4285-97d3-73c035c4f057" containerID="33eae27d867bc1cc56c0924b6ae56c71ce87c79e786951ec67d26648268996ce" exitCode=0 Sep 10 13:28:24.004650 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:24.004577 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atxlxt" event={"ID":"d707d982-f9bb-4285-97d3-73c035c4f057","Type":"ContainerDied","Data":"33eae27d867bc1cc56c0924b6ae56c71ce87c79e786951ec67d26648268996ce"} Sep 10 13:28:24.136800 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:24.136772 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdzs8" Sep 10 13:28:24.172302 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:24.172278 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zvhf2" Sep 10 13:28:24.207873 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:24.207838 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d362a34-f95f-41ad-b059-086c60b9c775-util\") pod \"7d362a34-f95f-41ad-b059-086c60b9c775\" (UID: \"7d362a34-f95f-41ad-b059-086c60b9c775\") " Sep 10 13:28:24.208062 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:24.207881 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/57831dfb-1fc9-4b0f-8c92-a9247c8a51ab-util\") pod \"57831dfb-1fc9-4b0f-8c92-a9247c8a51ab\" (UID: \"57831dfb-1fc9-4b0f-8c92-a9247c8a51ab\") " Sep 10 13:28:24.208062 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:24.207947 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jjkbm\" (UniqueName: \"kubernetes.io/projected/7d362a34-f95f-41ad-b059-086c60b9c775-kube-api-access-jjkbm\") pod \"7d362a34-f95f-41ad-b059-086c60b9c775\" (UID: \"7d362a34-f95f-41ad-b059-086c60b9c775\") " Sep 10 13:28:24.208062 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:24.207962 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/57831dfb-1fc9-4b0f-8c92-a9247c8a51ab-bundle\") pod \"57831dfb-1fc9-4b0f-8c92-a9247c8a51ab\" (UID: \"57831dfb-1fc9-4b0f-8c92-a9247c8a51ab\") " Sep 10 13:28:24.208062 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:24.208004 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mr52g\" (UniqueName: \"kubernetes.io/projected/57831dfb-1fc9-4b0f-8c92-a9247c8a51ab-kube-api-access-mr52g\") pod \"57831dfb-1fc9-4b0f-8c92-a9247c8a51ab\" (UID: \"57831dfb-1fc9-4b0f-8c92-a9247c8a51ab\") " Sep 10 13:28:24.208282 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:24.208141 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d362a34-f95f-41ad-b059-086c60b9c775-bundle\") pod \"7d362a34-f95f-41ad-b059-086c60b9c775\" (UID: \"7d362a34-f95f-41ad-b059-086c60b9c775\") " Sep 10 13:28:24.208761 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:24.208519 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57831dfb-1fc9-4b0f-8c92-a9247c8a51ab-bundle" (OuterVolumeSpecName: "bundle") pod "57831dfb-1fc9-4b0f-8c92-a9247c8a51ab" (UID: "57831dfb-1fc9-4b0f-8c92-a9247c8a51ab"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:28:24.208886 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:24.208834 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d362a34-f95f-41ad-b059-086c60b9c775-bundle" (OuterVolumeSpecName: "bundle") pod "7d362a34-f95f-41ad-b059-086c60b9c775" (UID: "7d362a34-f95f-41ad-b059-086c60b9c775"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:28:24.210359 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:24.210333 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d362a34-f95f-41ad-b059-086c60b9c775-kube-api-access-jjkbm" (OuterVolumeSpecName: "kube-api-access-jjkbm") pod "7d362a34-f95f-41ad-b059-086c60b9c775" (UID: "7d362a34-f95f-41ad-b059-086c60b9c775"). InnerVolumeSpecName "kube-api-access-jjkbm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:28:24.210533 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:24.210509 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57831dfb-1fc9-4b0f-8c92-a9247c8a51ab-kube-api-access-mr52g" (OuterVolumeSpecName: "kube-api-access-mr52g") pod "57831dfb-1fc9-4b0f-8c92-a9247c8a51ab" (UID: "57831dfb-1fc9-4b0f-8c92-a9247c8a51ab"). InnerVolumeSpecName "kube-api-access-mr52g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:28:24.213638 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:24.213613 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d362a34-f95f-41ad-b059-086c60b9c775-util" (OuterVolumeSpecName: "util") pod "7d362a34-f95f-41ad-b059-086c60b9c775" (UID: "7d362a34-f95f-41ad-b059-086c60b9c775"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:28:24.214590 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:24.214570 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57831dfb-1fc9-4b0f-8c92-a9247c8a51ab-util" (OuterVolumeSpecName: "util") pod "57831dfb-1fc9-4b0f-8c92-a9247c8a51ab" (UID: "57831dfb-1fc9-4b0f-8c92-a9247c8a51ab"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:28:24.309772 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:24.309665 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d362a34-f95f-41ad-b059-086c60b9c775-bundle\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:28:24.309772 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:24.309712 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d362a34-f95f-41ad-b059-086c60b9c775-util\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:28:24.309772 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:24.309721 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/57831dfb-1fc9-4b0f-8c92-a9247c8a51ab-util\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:28:24.309772 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:24.309729 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjkbm\" (UniqueName: \"kubernetes.io/projected/7d362a34-f95f-41ad-b059-086c60b9c775-kube-api-access-jjkbm\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:28:24.309772 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:24.309738 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/57831dfb-1fc9-4b0f-8c92-a9247c8a51ab-bundle\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:28:24.309772 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:24.309748 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mr52g\" (UniqueName: \"kubernetes.io/projected/57831dfb-1fc9-4b0f-8c92-a9247c8a51ab-kube-api-access-mr52g\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:28:25.011444 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:25.011396 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdzs8" Sep 10 13:28:25.011444 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:25.011426 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wdzs8" event={"ID":"7d362a34-f95f-41ad-b059-086c60b9c775","Type":"ContainerDied","Data":"a024422c6b8e2321e625a5dc494421685dfe44b27ce0652e973d88aa87aad6f3"} Sep 10 13:28:25.011924 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:25.011471 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a024422c6b8e2321e625a5dc494421685dfe44b27ce0652e973d88aa87aad6f3" Sep 10 13:28:25.013648 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:25.013602 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zvhf2" event={"ID":"57831dfb-1fc9-4b0f-8c92-a9247c8a51ab","Type":"ContainerDied","Data":"11882673c7720ac31216012f23b6af7ee9dc39e62295eb9feb1d0d026d5fb9d2"} Sep 10 13:28:25.013648 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:25.013639 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zvhf2" Sep 10 13:28:25.013842 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:25.013654 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="11882673c7720ac31216012f23b6af7ee9dc39e62295eb9feb1d0d026d5fb9d2" Sep 10 13:28:25.165542 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:25.165515 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5887h" Sep 10 13:28:25.168117 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:25.168096 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atxlxt" Sep 10 13:28:25.218953 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:25.218920 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b7e242b-86ea-463d-9ac7-e449db63a3a2-bundle\") pod \"0b7e242b-86ea-463d-9ac7-e449db63a3a2\" (UID: \"0b7e242b-86ea-463d-9ac7-e449db63a3a2\") " Sep 10 13:28:25.219155 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:25.218961 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hq5qz\" (UniqueName: \"kubernetes.io/projected/0b7e242b-86ea-463d-9ac7-e449db63a3a2-kube-api-access-hq5qz\") pod \"0b7e242b-86ea-463d-9ac7-e449db63a3a2\" (UID: \"0b7e242b-86ea-463d-9ac7-e449db63a3a2\") " Sep 10 13:28:25.219155 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:25.218978 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b7e242b-86ea-463d-9ac7-e449db63a3a2-util\") pod \"0b7e242b-86ea-463d-9ac7-e449db63a3a2\" (UID: \"0b7e242b-86ea-463d-9ac7-e449db63a3a2\") " Sep 10 13:28:25.219155 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:25.219069 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d707d982-f9bb-4285-97d3-73c035c4f057-util\") pod \"d707d982-f9bb-4285-97d3-73c035c4f057\" (UID: \"d707d982-f9bb-4285-97d3-73c035c4f057\") " Sep 10 13:28:25.219155 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:25.219098 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d707d982-f9bb-4285-97d3-73c035c4f057-bundle\") pod \"d707d982-f9bb-4285-97d3-73c035c4f057\" (UID: \"d707d982-f9bb-4285-97d3-73c035c4f057\") " Sep 10 13:28:25.219155 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:25.219116 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhjrg\" (UniqueName: \"kubernetes.io/projected/d707d982-f9bb-4285-97d3-73c035c4f057-kube-api-access-hhjrg\") pod \"d707d982-f9bb-4285-97d3-73c035c4f057\" (UID: \"d707d982-f9bb-4285-97d3-73c035c4f057\") " Sep 10 13:28:25.219644 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:25.219621 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b7e242b-86ea-463d-9ac7-e449db63a3a2-bundle" (OuterVolumeSpecName: "bundle") pod "0b7e242b-86ea-463d-9ac7-e449db63a3a2" (UID: "0b7e242b-86ea-463d-9ac7-e449db63a3a2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:28:25.219895 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:25.219869 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d707d982-f9bb-4285-97d3-73c035c4f057-bundle" (OuterVolumeSpecName: "bundle") pod "d707d982-f9bb-4285-97d3-73c035c4f057" (UID: "d707d982-f9bb-4285-97d3-73c035c4f057"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:28:25.221249 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:25.221223 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d707d982-f9bb-4285-97d3-73c035c4f057-kube-api-access-hhjrg" (OuterVolumeSpecName: "kube-api-access-hhjrg") pod "d707d982-f9bb-4285-97d3-73c035c4f057" (UID: "d707d982-f9bb-4285-97d3-73c035c4f057"). InnerVolumeSpecName "kube-api-access-hhjrg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:28:25.221500 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:25.221474 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b7e242b-86ea-463d-9ac7-e449db63a3a2-kube-api-access-hq5qz" (OuterVolumeSpecName: "kube-api-access-hq5qz") pod "0b7e242b-86ea-463d-9ac7-e449db63a3a2" (UID: "0b7e242b-86ea-463d-9ac7-e449db63a3a2"). InnerVolumeSpecName "kube-api-access-hq5qz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:28:25.224957 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:25.224933 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b7e242b-86ea-463d-9ac7-e449db63a3a2-util" (OuterVolumeSpecName: "util") pod "0b7e242b-86ea-463d-9ac7-e449db63a3a2" (UID: "0b7e242b-86ea-463d-9ac7-e449db63a3a2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:28:25.226364 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:25.226338 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d707d982-f9bb-4285-97d3-73c035c4f057-util" (OuterVolumeSpecName: "util") pod "d707d982-f9bb-4285-97d3-73c035c4f057" (UID: "d707d982-f9bb-4285-97d3-73c035c4f057"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:28:25.320819 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:25.320734 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d707d982-f9bb-4285-97d3-73c035c4f057-util\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:28:25.320819 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:25.320764 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d707d982-f9bb-4285-97d3-73c035c4f057-bundle\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:28:25.320819 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:25.320774 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhjrg\" (UniqueName: \"kubernetes.io/projected/d707d982-f9bb-4285-97d3-73c035c4f057-kube-api-access-hhjrg\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:28:25.320819 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:25.320783 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b7e242b-86ea-463d-9ac7-e449db63a3a2-bundle\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:28:25.320819 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:25.320791 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hq5qz\" (UniqueName: \"kubernetes.io/projected/0b7e242b-86ea-463d-9ac7-e449db63a3a2-kube-api-access-hq5qz\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:28:25.320819 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:25.320799 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b7e242b-86ea-463d-9ac7-e449db63a3a2-util\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:28:26.021815 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:26.021783 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5887h" Sep 10 13:28:26.021815 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:26.021801 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c5887h" event={"ID":"0b7e242b-86ea-463d-9ac7-e449db63a3a2","Type":"ContainerDied","Data":"e359dcac19cd7ddccee9a763a2f5ea247a3b611ef1804ab43e33ce118576b154"} Sep 10 13:28:26.022235 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:26.021847 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e359dcac19cd7ddccee9a763a2f5ea247a3b611ef1804ab43e33ce118576b154" Sep 10 13:28:26.023852 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:26.023830 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atxlxt" event={"ID":"d707d982-f9bb-4285-97d3-73c035c4f057","Type":"ContainerDied","Data":"ba602446824dfaba5e79e25ad0fb6eafbe2376eed7190e2373d50f45a0939a36"} Sep 10 13:28:26.023852 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:26.023847 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847atxlxt" Sep 10 13:28:26.024029 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:26.023865 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ba602446824dfaba5e79e25ad0fb6eafbe2376eed7190e2373d50f45a0939a36" Sep 10 13:28:26.245289 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:26.245253 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:28:31.371671 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:31.371629 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:28:32.181683 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:32.181647 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:28:34.887939 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:34.887899 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:28:34.993469 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:34.993385 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:28:35.565611 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:35.565573 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:28:37.043502 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:37.043461 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:28:37.843073 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:37.843040 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:28:39.556781 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:39.556749 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:28:39.644769 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:39.644731 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:28:40.192108 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:40.192073 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:28:40.608904 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:40.608817 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:28:40.879549 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:40.879452 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:28:41.017837 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:41.017804 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:28:45.378861 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:45.378824 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:28:45.485751 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:45.485714 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:28:45.835548 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:45.835509 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:28:46.029193 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:46.029160 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:28:49.576883 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:49.576849 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:28:49.677816 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:49.677766 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:28:51.007076 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:51.007039 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:28:51.167149 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:51.167111 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:28:51.286392 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:51.286306 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:28:57.351167 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:57.351121 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:28:58.719886 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:58.719841 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:28:58.831643 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:28:58.831610 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:29:05.931023 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:29:05.930971 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:29:06.090098 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:29:06.090064 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:29:06.394458 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:29:06.394349 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:29:06.605036 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:29:06.604999 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:29:06.866042 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:29:06.866010 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:29:10.339261 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:29:10.339224 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:29:10.484121 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:29:10.484086 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:29:12.408695 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:29:12.408656 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:29:12.553901 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:29:12.553869 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:29:18.085350 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:29:18.085312 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:29:18.182571 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:29:18.182530 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:29:18.310106 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:29:18.310063 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:29:18.432872 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:29:18.432839 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:29:35.416795 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:29:35.416757 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:29:35.712277 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:29:35.712171 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:29:40.242243 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:29:40.242206 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:29:40.373381 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:29:40.373346 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:29:40.479322 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:29:40.479286 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:29:41.194953 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:29:41.194920 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:29:41.301544 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:29:41.301507 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:30:01.545961 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:30:01.545926 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:30:01.684552 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:30:01.684515 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:30:02.237841 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:30:02.237806 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:30:11.065349 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:30:11.065298 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:30:16.579824 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:30:16.579791 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:30:16.795280 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:30:16.795243 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:30:24.518957 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:30:24.518905 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:30:24.703624 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:30:24.703589 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:30:24.859913 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:30:24.859828 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:30:25.037341 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:30:25.037298 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:30:28.400500 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:30:28.400444 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:30:28.510361 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:30:28.510323 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:31:00.197035 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:00.196993 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:31:02.589145 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.589111 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-f27mg"] Sep 10 13:31:02.589560 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.589468 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b7e242b-86ea-463d-9ac7-e449db63a3a2" containerName="pull" Sep 10 13:31:02.589560 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.589481 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b7e242b-86ea-463d-9ac7-e449db63a3a2" containerName="pull" Sep 10 13:31:02.589560 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.589489 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b7e242b-86ea-463d-9ac7-e449db63a3a2" containerName="extract" Sep 10 13:31:02.589560 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.589494 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b7e242b-86ea-463d-9ac7-e449db63a3a2" containerName="extract" Sep 10 13:31:02.589560 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.589504 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d362a34-f95f-41ad-b059-086c60b9c775" containerName="pull" Sep 10 13:31:02.589560 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.589509 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d362a34-f95f-41ad-b059-086c60b9c775" containerName="pull" Sep 10 13:31:02.589560 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.589516 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57831dfb-1fc9-4b0f-8c92-a9247c8a51ab" containerName="pull" Sep 10 13:31:02.589560 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.589521 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="57831dfb-1fc9-4b0f-8c92-a9247c8a51ab" containerName="pull" Sep 10 13:31:02.589560 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.589530 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57831dfb-1fc9-4b0f-8c92-a9247c8a51ab" containerName="util" Sep 10 13:31:02.589560 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.589535 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="57831dfb-1fc9-4b0f-8c92-a9247c8a51ab" containerName="util" Sep 10 13:31:02.589560 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.589544 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57831dfb-1fc9-4b0f-8c92-a9247c8a51ab" containerName="extract" Sep 10 13:31:02.589560 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.589549 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="57831dfb-1fc9-4b0f-8c92-a9247c8a51ab" containerName="extract" Sep 10 13:31:02.589560 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.589562 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d362a34-f95f-41ad-b059-086c60b9c775" containerName="util" Sep 10 13:31:02.589560 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.589567 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d362a34-f95f-41ad-b059-086c60b9c775" containerName="util" Sep 10 13:31:02.589980 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.589573 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d362a34-f95f-41ad-b059-086c60b9c775" containerName="extract" Sep 10 13:31:02.589980 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.589578 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d362a34-f95f-41ad-b059-086c60b9c775" containerName="extract" Sep 10 13:31:02.589980 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.589585 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b7e242b-86ea-463d-9ac7-e449db63a3a2" containerName="util" Sep 10 13:31:02.589980 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.589590 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b7e242b-86ea-463d-9ac7-e449db63a3a2" containerName="util" Sep 10 13:31:02.589980 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.589595 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d707d982-f9bb-4285-97d3-73c035c4f057" containerName="extract" Sep 10 13:31:02.589980 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.589601 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d707d982-f9bb-4285-97d3-73c035c4f057" containerName="extract" Sep 10 13:31:02.589980 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.589608 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d707d982-f9bb-4285-97d3-73c035c4f057" containerName="util" Sep 10 13:31:02.589980 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.589613 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d707d982-f9bb-4285-97d3-73c035c4f057" containerName="util" Sep 10 13:31:02.589980 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.589620 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d707d982-f9bb-4285-97d3-73c035c4f057" containerName="pull" Sep 10 13:31:02.589980 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.589625 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d707d982-f9bb-4285-97d3-73c035c4f057" containerName="pull" Sep 10 13:31:02.589980 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.589678 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b7e242b-86ea-463d-9ac7-e449db63a3a2" containerName="extract" Sep 10 13:31:02.589980 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.589687 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="57831dfb-1fc9-4b0f-8c92-a9247c8a51ab" containerName="extract" Sep 10 13:31:02.589980 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.589694 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d362a34-f95f-41ad-b059-086c60b9c775" containerName="extract" Sep 10 13:31:02.589980 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.589700 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d707d982-f9bb-4285-97d3-73c035c4f057" containerName="extract" Sep 10 13:31:02.592513 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.592490 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-f27mg" Sep 10 13:31:02.600863 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:31:02.600832 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"payload-processing-dockercfg-pcjc7\" is forbidden: User \"system:node:ip-10-0-142-118.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-142-118.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-pcjc7\"" type="*v1.Secret" Sep 10 13:31:02.600863 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.600850 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Sep 10 13:31:02.667043 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.666998 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztvsk\" (UniqueName: \"kubernetes.io/projected/88456670-7401-4573-9af4-58409e5bbd86-kube-api-access-ztvsk\") pod \"payload-pre-processing-59fbc47bf8-f27mg\" (UID: \"88456670-7401-4573-9af4-58409e5bbd86\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-f27mg" Sep 10 13:31:02.667213 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.667097 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88456670-7401-4573-9af4-58409e5bbd86-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-f27mg\" (UID: \"88456670-7401-4573-9af4-58409e5bbd86\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-f27mg" Sep 10 13:31:02.725078 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.725039 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-f27mg"] Sep 10 13:31:02.767916 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.767882 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88456670-7401-4573-9af4-58409e5bbd86-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-f27mg\" (UID: \"88456670-7401-4573-9af4-58409e5bbd86\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-f27mg" Sep 10 13:31:02.768134 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.767952 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztvsk\" (UniqueName: \"kubernetes.io/projected/88456670-7401-4573-9af4-58409e5bbd86-kube-api-access-ztvsk\") pod \"payload-pre-processing-59fbc47bf8-f27mg\" (UID: \"88456670-7401-4573-9af4-58409e5bbd86\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-f27mg" Sep 10 13:31:02.768683 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.768659 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88456670-7401-4573-9af4-58409e5bbd86-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-f27mg\" (UID: \"88456670-7401-4573-9af4-58409e5bbd86\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-f27mg" Sep 10 13:31:02.841968 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:02.841895 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztvsk\" (UniqueName: \"kubernetes.io/projected/88456670-7401-4573-9af4-58409e5bbd86-kube-api-access-ztvsk\") pod \"payload-pre-processing-59fbc47bf8-f27mg\" (UID: \"88456670-7401-4573-9af4-58409e5bbd86\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-f27mg" Sep 10 13:31:03.903607 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:03.903566 2560 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-pre-processing-59fbc47bf8-f27mg" secret="" err="failed to sync secret cache: timed out waiting for the condition" Sep 10 13:31:03.904016 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:03.903640 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-f27mg" Sep 10 13:31:04.160982 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:04.160906 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:31:04.200155 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:04.200126 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-f27mg"] Sep 10 13:31:04.202354 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:31:04.202331 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88456670_7401_4573_9af4_58409e5bbd86.slice/crio-e9f1e5c00a813f973d67038dd66cf7cbd3be99aa84f5eba8578dcb094d7399a4 WatchSource:0}: Error finding container e9f1e5c00a813f973d67038dd66cf7cbd3be99aa84f5eba8578dcb094d7399a4: Status 404 returned error can't find the container with id e9f1e5c00a813f973d67038dd66cf7cbd3be99aa84f5eba8578dcb094d7399a4 Sep 10 13:31:04.204230 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:04.204207 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-pcjc7\"" Sep 10 13:31:04.204696 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:04.204681 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 13:31:04.870660 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:04.870619 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-f27mg" event={"ID":"88456670-7401-4573-9af4-58409e5bbd86","Type":"ContainerStarted","Data":"e9f1e5c00a813f973d67038dd66cf7cbd3be99aa84f5eba8578dcb094d7399a4"} Sep 10 13:31:06.884820 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:06.884785 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-f27mg" event={"ID":"88456670-7401-4573-9af4-58409e5bbd86","Type":"ContainerStarted","Data":"44340b7333f9150d228159f87b7bad9494fb77230de495a0e9e67aa3f32ef5eb"} Sep 10 13:31:06.885242 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:06.884911 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-f27mg" Sep 10 13:31:06.917964 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:06.917908 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-f27mg" podStartSLOduration=2.449099316 podStartE2EDuration="4.917891718s" totalImagesPullingTime="2.468792402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:31:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:31:06.916716039 +0000 UTC m=+776.811433459" watchObservedRunningTime="2026-09-10 13:31:06.917891718 +0000 UTC m=+776.812609136" Sep 10 13:31:11.633216 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:11.633174 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:31:11.815227 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:11.815185 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:31:17.893559 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:17.893530 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-f27mg" Sep 10 13:31:24.433291 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:24.433254 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd"] Sep 10 13:31:24.436974 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:24.436955 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd" Sep 10 13:31:24.443716 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:24.443690 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Sep 10 13:31:24.444427 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:24.444385 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 10 13:31:24.444697 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:24.444678 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 10 13:31:24.444786 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:24.444758 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-nhrvj\"" Sep 10 13:31:24.538452 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:24.538396 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd"] Sep 10 13:31:24.544369 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:24.544329 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:31:24.560522 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:24.560490 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5ee4a41-7e88-44b8-ac33-879e10aca6a0-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd\" (UID: \"e5ee4a41-7e88-44b8-ac33-879e10aca6a0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd" Sep 10 13:31:24.560522 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:24.560521 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5ee4a41-7e88-44b8-ac33-879e10aca6a0-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd\" (UID: \"e5ee4a41-7e88-44b8-ac33-879e10aca6a0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd" Sep 10 13:31:24.560732 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:24.560555 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5ee4a41-7e88-44b8-ac33-879e10aca6a0-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd\" (UID: \"e5ee4a41-7e88-44b8-ac33-879e10aca6a0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd" Sep 10 13:31:24.560732 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:24.560581 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5ee4a41-7e88-44b8-ac33-879e10aca6a0-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd\" (UID: \"e5ee4a41-7e88-44b8-ac33-879e10aca6a0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd" Sep 10 13:31:24.560732 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:24.560615 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtf9n\" (UniqueName: \"kubernetes.io/projected/e5ee4a41-7e88-44b8-ac33-879e10aca6a0-kube-api-access-gtf9n\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd\" (UID: \"e5ee4a41-7e88-44b8-ac33-879e10aca6a0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd" Sep 10 13:31:24.560732 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:24.560683 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5ee4a41-7e88-44b8-ac33-879e10aca6a0-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd\" (UID: \"e5ee4a41-7e88-44b8-ac33-879e10aca6a0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd" Sep 10 13:31:24.661699 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:24.661665 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5ee4a41-7e88-44b8-ac33-879e10aca6a0-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd\" (UID: \"e5ee4a41-7e88-44b8-ac33-879e10aca6a0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd" Sep 10 13:31:24.661699 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:24.661703 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5ee4a41-7e88-44b8-ac33-879e10aca6a0-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd\" (UID: \"e5ee4a41-7e88-44b8-ac33-879e10aca6a0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd" Sep 10 13:31:24.661972 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:24.661886 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtf9n\" (UniqueName: \"kubernetes.io/projected/e5ee4a41-7e88-44b8-ac33-879e10aca6a0-kube-api-access-gtf9n\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd\" (UID: \"e5ee4a41-7e88-44b8-ac33-879e10aca6a0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd" Sep 10 13:31:24.661972 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:24.661957 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5ee4a41-7e88-44b8-ac33-879e10aca6a0-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd\" (UID: \"e5ee4a41-7e88-44b8-ac33-879e10aca6a0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd" Sep 10 13:31:24.662086 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:24.662008 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5ee4a41-7e88-44b8-ac33-879e10aca6a0-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd\" (UID: \"e5ee4a41-7e88-44b8-ac33-879e10aca6a0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd" Sep 10 13:31:24.662086 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:24.662036 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5ee4a41-7e88-44b8-ac33-879e10aca6a0-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd\" (UID: \"e5ee4a41-7e88-44b8-ac33-879e10aca6a0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd" Sep 10 13:31:24.662222 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:24.662110 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5ee4a41-7e88-44b8-ac33-879e10aca6a0-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd\" (UID: \"e5ee4a41-7e88-44b8-ac33-879e10aca6a0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd" Sep 10 13:31:24.662281 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:24.662229 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5ee4a41-7e88-44b8-ac33-879e10aca6a0-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd\" (UID: \"e5ee4a41-7e88-44b8-ac33-879e10aca6a0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd" Sep 10 13:31:24.662395 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:24.662367 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5ee4a41-7e88-44b8-ac33-879e10aca6a0-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd\" (UID: \"e5ee4a41-7e88-44b8-ac33-879e10aca6a0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd" Sep 10 13:31:24.664276 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:24.664254 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5ee4a41-7e88-44b8-ac33-879e10aca6a0-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd\" (UID: \"e5ee4a41-7e88-44b8-ac33-879e10aca6a0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd" Sep 10 13:31:24.664614 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:24.664594 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5ee4a41-7e88-44b8-ac33-879e10aca6a0-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd\" (UID: \"e5ee4a41-7e88-44b8-ac33-879e10aca6a0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd" Sep 10 13:31:24.675466 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:24.675437 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtf9n\" (UniqueName: \"kubernetes.io/projected/e5ee4a41-7e88-44b8-ac33-879e10aca6a0-kube-api-access-gtf9n\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd\" (UID: \"e5ee4a41-7e88-44b8-ac33-879e10aca6a0\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd" Sep 10 13:31:24.747317 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:24.747224 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd" Sep 10 13:31:25.060264 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:25.060225 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd"] Sep 10 13:31:25.060264 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:31:25.060236 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5ee4a41_7e88_44b8_ac33_879e10aca6a0.slice/crio-2988812c86fbf760b089a5f891a58085f73008d84663d3e7667a1a4b041e0447 WatchSource:0}: Error finding container 2988812c86fbf760b089a5f891a58085f73008d84663d3e7667a1a4b041e0447: Status 404 returned error can't find the container with id 2988812c86fbf760b089a5f891a58085f73008d84663d3e7667a1a4b041e0447 Sep 10 13:31:25.986276 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:25.986235 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd" event={"ID":"e5ee4a41-7e88-44b8-ac33-879e10aca6a0","Type":"ContainerStarted","Data":"2988812c86fbf760b089a5f891a58085f73008d84663d3e7667a1a4b041e0447"} Sep 10 13:31:26.993050 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:26.993013 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd" event={"ID":"e5ee4a41-7e88-44b8-ac33-879e10aca6a0","Type":"ContainerStarted","Data":"39ee2b7f076402db685d0465f6916c81a76a17777fae71e93e828665f6907028"} Sep 10 13:31:27.069232 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:27.068562 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd" podStartSLOduration=1.9335923149999998 podStartE2EDuration="3.06853881s" totalImagesPullingTime="1.134946495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:31:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:31:27.066224035 +0000 UTC m=+796.960941454" watchObservedRunningTime="2026-09-10 13:31:27.06853881 +0000 UTC m=+796.963256232" Sep 10 13:31:29.291219 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:29.291177 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:31:34.747459 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:34.747414 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd" Sep 10 13:31:34.747975 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:34.747479 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd" Sep 10 13:31:34.760052 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:34.760025 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd" Sep 10 13:31:35.048210 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:35.048130 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd" Sep 10 13:31:41.578214 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:41.578172 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:31:47.366373 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:47.366334 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:31:53.721804 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:53.721767 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-ggln2"] Sep 10 13:31:53.727846 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:53.727818 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ggln2" Sep 10 13:31:53.731349 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:53.731327 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Sep 10 13:31:53.737102 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:53.737077 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-ggln2"] Sep 10 13:31:53.774145 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:53.774111 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:31:53.828126 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:53.828094 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2940f367-8823-4044-b6dd-29d08ad299f0-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ggln2\" (UID: \"2940f367-8823-4044-b6dd-29d08ad299f0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ggln2" Sep 10 13:31:53.828313 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:53.828134 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2940f367-8823-4044-b6dd-29d08ad299f0-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ggln2\" (UID: \"2940f367-8823-4044-b6dd-29d08ad299f0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ggln2" Sep 10 13:31:53.828313 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:53.828160 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2940f367-8823-4044-b6dd-29d08ad299f0-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ggln2\" (UID: \"2940f367-8823-4044-b6dd-29d08ad299f0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ggln2" Sep 10 13:31:53.828313 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:53.828241 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2940f367-8823-4044-b6dd-29d08ad299f0-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ggln2\" (UID: \"2940f367-8823-4044-b6dd-29d08ad299f0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ggln2" Sep 10 13:31:53.828313 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:53.828284 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2940f367-8823-4044-b6dd-29d08ad299f0-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ggln2\" (UID: \"2940f367-8823-4044-b6dd-29d08ad299f0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ggln2" Sep 10 13:31:53.828513 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:53.828320 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hd4gb\" (UniqueName: \"kubernetes.io/projected/2940f367-8823-4044-b6dd-29d08ad299f0-kube-api-access-hd4gb\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ggln2\" (UID: \"2940f367-8823-4044-b6dd-29d08ad299f0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ggln2" Sep 10 13:31:53.929551 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:53.929513 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2940f367-8823-4044-b6dd-29d08ad299f0-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ggln2\" (UID: \"2940f367-8823-4044-b6dd-29d08ad299f0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ggln2" Sep 10 13:31:53.929744 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:53.929568 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2940f367-8823-4044-b6dd-29d08ad299f0-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ggln2\" (UID: \"2940f367-8823-4044-b6dd-29d08ad299f0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ggln2" Sep 10 13:31:53.929744 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:53.929599 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hd4gb\" (UniqueName: \"kubernetes.io/projected/2940f367-8823-4044-b6dd-29d08ad299f0-kube-api-access-hd4gb\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ggln2\" (UID: \"2940f367-8823-4044-b6dd-29d08ad299f0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ggln2" Sep 10 13:31:53.929744 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:53.929682 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2940f367-8823-4044-b6dd-29d08ad299f0-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ggln2\" (UID: \"2940f367-8823-4044-b6dd-29d08ad299f0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ggln2" Sep 10 13:31:53.929744 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:53.929718 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2940f367-8823-4044-b6dd-29d08ad299f0-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ggln2\" (UID: \"2940f367-8823-4044-b6dd-29d08ad299f0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ggln2" Sep 10 13:31:53.929744 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:53.929745 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2940f367-8823-4044-b6dd-29d08ad299f0-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ggln2\" (UID: \"2940f367-8823-4044-b6dd-29d08ad299f0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ggln2" Sep 10 13:31:53.930041 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:53.929978 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2940f367-8823-4044-b6dd-29d08ad299f0-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ggln2\" (UID: \"2940f367-8823-4044-b6dd-29d08ad299f0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ggln2" Sep 10 13:31:53.930041 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:53.930012 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2940f367-8823-4044-b6dd-29d08ad299f0-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ggln2\" (UID: \"2940f367-8823-4044-b6dd-29d08ad299f0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ggln2" Sep 10 13:31:53.930237 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:53.930217 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2940f367-8823-4044-b6dd-29d08ad299f0-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ggln2\" (UID: \"2940f367-8823-4044-b6dd-29d08ad299f0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ggln2" Sep 10 13:31:53.932264 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:53.932237 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2940f367-8823-4044-b6dd-29d08ad299f0-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ggln2\" (UID: \"2940f367-8823-4044-b6dd-29d08ad299f0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ggln2" Sep 10 13:31:53.932668 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:53.932647 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2940f367-8823-4044-b6dd-29d08ad299f0-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ggln2\" (UID: \"2940f367-8823-4044-b6dd-29d08ad299f0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ggln2" Sep 10 13:31:53.943513 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:53.943485 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hd4gb\" (UniqueName: \"kubernetes.io/projected/2940f367-8823-4044-b6dd-29d08ad299f0-kube-api-access-hd4gb\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ggln2\" (UID: \"2940f367-8823-4044-b6dd-29d08ad299f0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ggln2" Sep 10 13:31:54.039587 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:54.039491 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ggln2" Sep 10 13:31:54.328050 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:54.328014 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-ggln2"] Sep 10 13:31:54.328354 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:31:54.328327 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2940f367_8823_4044_b6dd_29d08ad299f0.slice/crio-67c6fec14d5d3327273794c8ea53f58c53d9b363c08a95e65fdc795b5d1ed92c WatchSource:0}: Error finding container 67c6fec14d5d3327273794c8ea53f58c53d9b363c08a95e65fdc795b5d1ed92c: Status 404 returned error can't find the container with id 67c6fec14d5d3327273794c8ea53f58c53d9b363c08a95e65fdc795b5d1ed92c Sep 10 13:31:55.146240 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:55.146202 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ggln2" event={"ID":"2940f367-8823-4044-b6dd-29d08ad299f0","Type":"ContainerStarted","Data":"5fbe1fda492b85ab13c378db936fdb3c925cd7f61aee3ff35007cb3a49300158"} Sep 10 13:31:55.146646 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:55.146256 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ggln2" event={"ID":"2940f367-8823-4044-b6dd-29d08ad299f0","Type":"ContainerStarted","Data":"67c6fec14d5d3327273794c8ea53f58c53d9b363c08a95e65fdc795b5d1ed92c"} Sep 10 13:31:55.185960 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:31:55.185899 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ggln2" podStartSLOduration=1.985102757 podStartE2EDuration="2.185882383s" totalImagesPullingTime="200.779626ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:31:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:31:55.181012707 +0000 UTC m=+825.075730136" watchObservedRunningTime="2026-09-10 13:31:55.185882383 +0000 UTC m=+825.080599817" Sep 10 13:32:03.716662 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:32:03.716623 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:32:03.809933 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:32:03.809902 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:32:04.039843 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:32:04.039742 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ggln2" Sep 10 13:32:04.039843 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:32:04.039816 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ggln2" Sep 10 13:32:04.052830 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:32:04.052802 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ggln2" Sep 10 13:32:04.207572 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:32:04.207545 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ggln2" Sep 10 13:32:08.058315 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:32:08.058275 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:32:08.262634 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:32:08.262596 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:32:15.751516 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:32:15.751478 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:32:20.960590 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:32:20.960555 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:33:10.508352 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:33:10.508319 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gtqlh_52fceedd-00f2-4943-ae47-dec51216d8e4/console-operator/2.log" Sep 10 13:33:10.510452 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:33:10.510431 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gtqlh_52fceedd-00f2-4943-ae47-dec51216d8e4/console-operator/2.log" Sep 10 13:34:14.201022 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:14.200991 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:14.374292 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:14.374258 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:15.510487 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:15.510451 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:15.742811 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:15.742776 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:31.544322 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:31.544285 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:31.671801 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:31.671763 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:31.976950 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:31.976917 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp"] Sep 10 13:34:31.980464 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:31.980445 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:34:31.983569 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:31.983549 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-bc03be-openshift-default-dockercfg-mlczj\"" Sep 10 13:34:32.010432 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:32.010379 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp"] Sep 10 13:34:32.056980 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:32.056949 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76l6h\" (UniqueName: \"kubernetes.io/projected/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-kube-api-access-76l6h\") pod \"e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp\" (UID: \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\") " pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:34:32.057152 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:32.056997 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-credential-socket\") pod \"e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp\" (UID: \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\") " pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:34:32.057152 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:32.057022 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-istio-token\") pod \"e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp\" (UID: \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\") " pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:34:32.057152 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:32.057063 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-workload-certs\") pod \"e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp\" (UID: \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\") " pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:34:32.057152 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:32.057085 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-istio-envoy\") pod \"e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp\" (UID: \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\") " pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:34:32.057300 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:32.057146 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-istio-data\") pod \"e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp\" (UID: \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\") " pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:34:32.057300 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:32.057180 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-istiod-ca-cert\") pod \"e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp\" (UID: \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\") " pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:34:32.057300 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:32.057205 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-workload-socket\") pod \"e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp\" (UID: \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\") " pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:34:32.057300 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:32.057230 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-istio-podinfo\") pod \"e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp\" (UID: \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\") " pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:34:32.106185 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:32.106149 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:32.158161 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:32.158125 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-istio-envoy\") pod \"e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp\" (UID: \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\") " pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:34:32.158343 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:32.158175 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-istio-data\") pod \"e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp\" (UID: \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\") " pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:34:32.158343 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:32.158229 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-istiod-ca-cert\") pod \"e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp\" (UID: \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\") " pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:34:32.158343 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:32.158256 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-workload-socket\") pod \"e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp\" (UID: \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\") " pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:34:32.158343 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:32.158280 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-istio-podinfo\") pod \"e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp\" (UID: \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\") " pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:34:32.158343 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:32.158341 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76l6h\" (UniqueName: \"kubernetes.io/projected/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-kube-api-access-76l6h\") pod \"e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp\" (UID: \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\") " pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:34:32.158637 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:32.158533 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-credential-socket\") pod \"e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp\" (UID: \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\") " pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:34:32.158637 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:32.158588 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-istio-token\") pod \"e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp\" (UID: \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\") " pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:34:32.158748 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:32.158650 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-workload-certs\") pod \"e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp\" (UID: \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\") " pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:34:32.158748 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:32.158670 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-istio-data\") pod \"e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp\" (UID: \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\") " pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:34:32.158748 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:32.158730 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-workload-socket\") pod \"e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp\" (UID: \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\") " pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:34:32.158958 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:32.158935 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-credential-socket\") pod \"e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp\" (UID: \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\") " pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:34:32.159030 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:32.159012 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-workload-certs\") pod \"e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp\" (UID: \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\") " pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:34:32.159123 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:32.159101 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-istiod-ca-cert\") pod \"e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp\" (UID: \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\") " pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:34:32.160563 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:32.160542 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-istio-envoy\") pod \"e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp\" (UID: \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\") " pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:34:32.160886 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:32.160861 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-istio-podinfo\") pod \"e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp\" (UID: \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\") " pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:34:32.179725 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:32.179699 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-istio-token\") pod \"e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp\" (UID: \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\") " pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:34:32.186515 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:32.186481 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76l6h\" (UniqueName: \"kubernetes.io/projected/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-kube-api-access-76l6h\") pod \"e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp\" (UID: \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\") " pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:34:32.251626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:32.251545 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:32.290484 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:32.290451 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:34:32.576228 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:32.576202 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp"] Sep 10 13:34:32.578611 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:34:32.578587 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef2fd3f9_1e16_4c56_95da_af9379f5ec52.slice/crio-503f0d4fb045cc1976f553e543c7dbbb72c45c7a89e422b8aa0336cdc30a3ac4 WatchSource:0}: Error finding container 503f0d4fb045cc1976f553e543c7dbbb72c45c7a89e422b8aa0336cdc30a3ac4: Status 404 returned error can't find the container with id 503f0d4fb045cc1976f553e543c7dbbb72c45c7a89e422b8aa0336cdc30a3ac4 Sep 10 13:34:32.580707 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:32.580677 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:34:32.580798 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:32.580741 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:34:32.580798 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:32.580766 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:34:32.957367 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:32.957331 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:33.018628 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:33.018595 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" event={"ID":"ef2fd3f9-1e16-4c56-95da-af9379f5ec52","Type":"ContainerStarted","Data":"3d1df39b4bf4f7bb9d2e3a0a57871be6c046aff9432557f63e4d17ee9b09cada"} Sep 10 13:34:33.018835 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:33.018648 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" event={"ID":"ef2fd3f9-1e16-4c56-95da-af9379f5ec52","Type":"ContainerStarted","Data":"503f0d4fb045cc1976f553e543c7dbbb72c45c7a89e422b8aa0336cdc30a3ac4"} Sep 10 13:34:33.056023 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:33.055970 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" podStartSLOduration=2.055947386 podStartE2EDuration="2.055947386s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:34:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:34:33.052220569 +0000 UTC m=+982.946937987" watchObservedRunningTime="2026-09-10 13:34:33.055947386 +0000 UTC m=+982.950664805" Sep 10 13:34:33.067359 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:33.067327 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:33.291457 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:33.291349 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:34:33.292809 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:33.292783 2560 patch_prober.go:28] interesting pod/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Sep 10 13:34:33.292899 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:33.292842 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" podUID="ef2fd3f9-1e16-4c56-95da-af9379f5ec52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Sep 10 13:34:33.432086 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:33.432045 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:34.295108 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:34.295075 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:34:35.029704 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:35.029670 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:34:35.030735 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:35.030715 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:34:35.100288 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:35.100252 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:35.276225 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:35.276192 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:36.086949 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:36.086907 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:36.507309 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:36.507271 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:36.606429 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:36.606370 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:36.725372 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:36.725337 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:37.383095 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:37.383057 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:37.499054 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:37.499016 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:37.594056 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:37.594012 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-bc03be-795f8655f4-w88qp"] Sep 10 13:34:37.597598 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:37.597572 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bc03be-795f8655f4-w88qp" Sep 10 13:34:37.601989 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:37.601967 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-bc03be\"" Sep 10 13:34:37.605360 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:37.605343 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-bc03be-dockercfg-pcqfb\"" Sep 10 13:34:37.622585 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:37.622554 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-bc03be-795f8655f4-w88qp"] Sep 10 13:34:37.698466 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:37.698425 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-bc03be-58d4c9b479-rqw4t"] Sep 10 13:34:37.701847 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:37.701829 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-bc03be-58d4c9b479-rqw4t" Sep 10 13:34:37.707564 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:37.707528 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szjmd\" (UniqueName: \"kubernetes.io/projected/9149a82b-029e-447f-a5ee-af1930e141a1-kube-api-access-szjmd\") pod \"payload-pre-processing-e2e-inf-a-bc03be-795f8655f4-w88qp\" (UID: \"9149a82b-029e-447f-a5ee-af1930e141a1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bc03be-795f8655f4-w88qp" Sep 10 13:34:37.707664 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:37.707592 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9149a82b-029e-447f-a5ee-af1930e141a1-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-bc03be-795f8655f4-w88qp\" (UID: \"9149a82b-029e-447f-a5ee-af1930e141a1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bc03be-795f8655f4-w88qp" Sep 10 13:34:37.718018 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:37.717990 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-bc03be-58d4c9b479-rqw4t"] Sep 10 13:34:37.808611 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:37.808566 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szjmd\" (UniqueName: \"kubernetes.io/projected/9149a82b-029e-447f-a5ee-af1930e141a1-kube-api-access-szjmd\") pod \"payload-pre-processing-e2e-inf-a-bc03be-795f8655f4-w88qp\" (UID: \"9149a82b-029e-447f-a5ee-af1930e141a1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bc03be-795f8655f4-w88qp" Sep 10 13:34:37.808825 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:37.808664 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9149a82b-029e-447f-a5ee-af1930e141a1-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-bc03be-795f8655f4-w88qp\" (UID: \"9149a82b-029e-447f-a5ee-af1930e141a1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bc03be-795f8655f4-w88qp" Sep 10 13:34:37.808825 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:37.808696 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxtqd\" (UniqueName: \"kubernetes.io/projected/27278e4d-9529-4f44-965e-e0a41074f5e3-kube-api-access-sxtqd\") pod \"payload-processing-e2e-inf-a-bc03be-58d4c9b479-rqw4t\" (UID: \"27278e4d-9529-4f44-965e-e0a41074f5e3\") " pod="openshift-ingress/payload-processing-e2e-inf-a-bc03be-58d4c9b479-rqw4t" Sep 10 13:34:37.808825 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:37.808766 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27278e4d-9529-4f44-965e-e0a41074f5e3-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-bc03be-58d4c9b479-rqw4t\" (UID: \"27278e4d-9529-4f44-965e-e0a41074f5e3\") " pod="openshift-ingress/payload-processing-e2e-inf-a-bc03be-58d4c9b479-rqw4t" Sep 10 13:34:37.809391 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:37.809371 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9149a82b-029e-447f-a5ee-af1930e141a1-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-bc03be-795f8655f4-w88qp\" (UID: \"9149a82b-029e-447f-a5ee-af1930e141a1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bc03be-795f8655f4-w88qp" Sep 10 13:34:37.819575 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:37.819555 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szjmd\" (UniqueName: \"kubernetes.io/projected/9149a82b-029e-447f-a5ee-af1930e141a1-kube-api-access-szjmd\") pod \"payload-pre-processing-e2e-inf-a-bc03be-795f8655f4-w88qp\" (UID: \"9149a82b-029e-447f-a5ee-af1930e141a1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bc03be-795f8655f4-w88qp" Sep 10 13:34:37.909072 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:37.909030 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bc03be-795f8655f4-w88qp" Sep 10 13:34:37.909309 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:37.909284 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxtqd\" (UniqueName: \"kubernetes.io/projected/27278e4d-9529-4f44-965e-e0a41074f5e3-kube-api-access-sxtqd\") pod \"payload-processing-e2e-inf-a-bc03be-58d4c9b479-rqw4t\" (UID: \"27278e4d-9529-4f44-965e-e0a41074f5e3\") " pod="openshift-ingress/payload-processing-e2e-inf-a-bc03be-58d4c9b479-rqw4t" Sep 10 13:34:37.909376 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:37.909340 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27278e4d-9529-4f44-965e-e0a41074f5e3-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-bc03be-58d4c9b479-rqw4t\" (UID: \"27278e4d-9529-4f44-965e-e0a41074f5e3\") " pod="openshift-ingress/payload-processing-e2e-inf-a-bc03be-58d4c9b479-rqw4t" Sep 10 13:34:37.910050 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:37.910032 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27278e4d-9529-4f44-965e-e0a41074f5e3-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-bc03be-58d4c9b479-rqw4t\" (UID: \"27278e4d-9529-4f44-965e-e0a41074f5e3\") " pod="openshift-ingress/payload-processing-e2e-inf-a-bc03be-58d4c9b479-rqw4t" Sep 10 13:34:37.918620 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:37.918590 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxtqd\" (UniqueName: \"kubernetes.io/projected/27278e4d-9529-4f44-965e-e0a41074f5e3-kube-api-access-sxtqd\") pod \"payload-processing-e2e-inf-a-bc03be-58d4c9b479-rqw4t\" (UID: \"27278e4d-9529-4f44-965e-e0a41074f5e3\") " pod="openshift-ingress/payload-processing-e2e-inf-a-bc03be-58d4c9b479-rqw4t" Sep 10 13:34:38.011728 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:38.011686 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-bc03be-58d4c9b479-rqw4t" Sep 10 13:34:38.194922 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:38.194897 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-bc03be-795f8655f4-w88qp"] Sep 10 13:34:38.197174 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:34:38.197149 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9149a82b_029e_447f_a5ee_af1930e141a1.slice/crio-87aab48a4917f9bc6f1792248e0d017f50487cd4986b40bac98bd842e1aa28b4 WatchSource:0}: Error finding container 87aab48a4917f9bc6f1792248e0d017f50487cd4986b40bac98bd842e1aa28b4: Status 404 returned error can't find the container with id 87aab48a4917f9bc6f1792248e0d017f50487cd4986b40bac98bd842e1aa28b4 Sep 10 13:34:38.296092 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:38.296055 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-bc03be-58d4c9b479-rqw4t"] Sep 10 13:34:38.297204 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:34:38.297172 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27278e4d_9529_4f44_965e_e0a41074f5e3.slice/crio-54ad281b47f05a4d176435090a25b615934b0c9d4f5e7ee4bfaf7d9d0ba54fcc WatchSource:0}: Error finding container 54ad281b47f05a4d176435090a25b615934b0c9d4f5e7ee4bfaf7d9d0ba54fcc: Status 404 returned error can't find the container with id 54ad281b47f05a4d176435090a25b615934b0c9d4f5e7ee4bfaf7d9d0ba54fcc Sep 10 13:34:39.061497 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:39.061455 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-bc03be-58d4c9b479-rqw4t" event={"ID":"27278e4d-9529-4f44-965e-e0a41074f5e3","Type":"ContainerStarted","Data":"3a696f9498ba651149dc841b0cf05ea7c2ab970f0042e2e4019bd84bf10432d7"} Sep 10 13:34:39.062065 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:39.061516 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-bc03be-58d4c9b479-rqw4t" event={"ID":"27278e4d-9529-4f44-965e-e0a41074f5e3","Type":"ContainerStarted","Data":"54ad281b47f05a4d176435090a25b615934b0c9d4f5e7ee4bfaf7d9d0ba54fcc"} Sep 10 13:34:39.062065 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:39.061699 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-bc03be-58d4c9b479-rqw4t" Sep 10 13:34:39.063219 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:39.063196 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bc03be-795f8655f4-w88qp" event={"ID":"9149a82b-029e-447f-a5ee-af1930e141a1","Type":"ContainerStarted","Data":"5be47c7dcd9799a96b7562c027916f562fa487966decbaf925f3444642e596bc"} Sep 10 13:34:39.063330 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:39.063236 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bc03be-795f8655f4-w88qp" event={"ID":"9149a82b-029e-447f-a5ee-af1930e141a1","Type":"ContainerStarted","Data":"87aab48a4917f9bc6f1792248e0d017f50487cd4986b40bac98bd842e1aa28b4"} Sep 10 13:34:39.063330 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:39.063270 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bc03be-795f8655f4-w88qp" Sep 10 13:34:39.085236 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:39.085181 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-bc03be-58d4c9b479-rqw4t" podStartSLOduration=2.08516419 podStartE2EDuration="2.08516419s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:34:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:34:39.08314681 +0000 UTC m=+988.977864226" watchObservedRunningTime="2026-09-10 13:34:39.08516419 +0000 UTC m=+988.979881661" Sep 10 13:34:39.110707 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:39.110656 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bc03be-795f8655f4-w88qp" podStartSLOduration=2.110642182 podStartE2EDuration="2.110642182s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:34:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:34:39.108113119 +0000 UTC m=+989.002830547" watchObservedRunningTime="2026-09-10 13:34:39.110642182 +0000 UTC m=+989.005359599" Sep 10 13:34:39.764033 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:39.764002 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:40.225945 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:40.225909 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:40.369113 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:40.369083 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:42.167975 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:42.167942 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:42.382173 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:42.382138 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:42.512960 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:42.512873 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:42.785608 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:42.785508 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:43.002391 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:43.002353 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:43.735239 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:43.735197 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:43.826340 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:43.826306 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:45.031311 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:45.031269 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:45.160621 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:45.160576 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:45.267656 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:45.267612 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:45.439121 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:45.439091 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:45.875172 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:45.875080 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:46.019541 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:46.019506 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:46.169086 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:46.169043 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-deba50-679c68dccf-g2wgd"] Sep 10 13:34:46.174796 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:46.174769 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-deba50-679c68dccf-g2wgd" Sep 10 13:34:46.181105 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:46.181030 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-deba50-dockercfg-x6dxj\"" Sep 10 13:34:46.181342 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:46.181326 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-deba50\"" Sep 10 13:34:46.202124 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:46.202093 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-deba50-679c68dccf-g2wgd"] Sep 10 13:34:46.293846 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:46.293804 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/225727ab-44d8-4879-bee7-70a5e2668cfb-plugins-config-volume\") pod \"payload-processing-e2e-autores-deba50-679c68dccf-g2wgd\" (UID: \"225727ab-44d8-4879-bee7-70a5e2668cfb\") " pod="openshift-ingress/payload-processing-e2e-autores-deba50-679c68dccf-g2wgd" Sep 10 13:34:46.294032 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:46.293916 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9nzm\" (UniqueName: \"kubernetes.io/projected/225727ab-44d8-4879-bee7-70a5e2668cfb-kube-api-access-b9nzm\") pod \"payload-processing-e2e-autores-deba50-679c68dccf-g2wgd\" (UID: \"225727ab-44d8-4879-bee7-70a5e2668cfb\") " pod="openshift-ingress/payload-processing-e2e-autores-deba50-679c68dccf-g2wgd" Sep 10 13:34:46.394392 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:46.394354 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/225727ab-44d8-4879-bee7-70a5e2668cfb-plugins-config-volume\") pod \"payload-processing-e2e-autores-deba50-679c68dccf-g2wgd\" (UID: \"225727ab-44d8-4879-bee7-70a5e2668cfb\") " pod="openshift-ingress/payload-processing-e2e-autores-deba50-679c68dccf-g2wgd" Sep 10 13:34:46.394625 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:46.394464 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9nzm\" (UniqueName: \"kubernetes.io/projected/225727ab-44d8-4879-bee7-70a5e2668cfb-kube-api-access-b9nzm\") pod \"payload-processing-e2e-autores-deba50-679c68dccf-g2wgd\" (UID: \"225727ab-44d8-4879-bee7-70a5e2668cfb\") " pod="openshift-ingress/payload-processing-e2e-autores-deba50-679c68dccf-g2wgd" Sep 10 13:34:46.395149 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:46.395124 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/225727ab-44d8-4879-bee7-70a5e2668cfb-plugins-config-volume\") pod \"payload-processing-e2e-autores-deba50-679c68dccf-g2wgd\" (UID: \"225727ab-44d8-4879-bee7-70a5e2668cfb\") " pod="openshift-ingress/payload-processing-e2e-autores-deba50-679c68dccf-g2wgd" Sep 10 13:34:46.408061 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:46.408026 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9nzm\" (UniqueName: \"kubernetes.io/projected/225727ab-44d8-4879-bee7-70a5e2668cfb-kube-api-access-b9nzm\") pod \"payload-processing-e2e-autores-deba50-679c68dccf-g2wgd\" (UID: \"225727ab-44d8-4879-bee7-70a5e2668cfb\") " pod="openshift-ingress/payload-processing-e2e-autores-deba50-679c68dccf-g2wgd" Sep 10 13:34:46.485854 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:46.485758 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-deba50-679c68dccf-g2wgd" Sep 10 13:34:46.578308 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:46.578223 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:46.765483 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:46.763316 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:46.779334 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:46.779313 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-deba50-679c68dccf-g2wgd"] Sep 10 13:34:46.782035 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:34:46.782005 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod225727ab_44d8_4879_bee7_70a5e2668cfb.slice/crio-9ef6633fa1fb4a530aa56b3a0004e2235ee100b4a8974ca3222b30815eebf148 WatchSource:0}: Error finding container 9ef6633fa1fb4a530aa56b3a0004e2235ee100b4a8974ca3222b30815eebf148: Status 404 returned error can't find the container with id 9ef6633fa1fb4a530aa56b3a0004e2235ee100b4a8974ca3222b30815eebf148 Sep 10 13:34:46.882220 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:46.882192 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:47.112317 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:47.112202 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-deba50-679c68dccf-g2wgd" event={"ID":"225727ab-44d8-4879-bee7-70a5e2668cfb","Type":"ContainerStarted","Data":"e45feffd4ff81c0a6e0a8a4c3328339cb90c8daccd07f0d4bfe72f7696864dd4"} Sep 10 13:34:47.112530 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:47.112357 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-deba50-679c68dccf-g2wgd" Sep 10 13:34:47.112530 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:47.112397 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-deba50-679c68dccf-g2wgd" event={"ID":"225727ab-44d8-4879-bee7-70a5e2668cfb","Type":"ContainerStarted","Data":"9ef6633fa1fb4a530aa56b3a0004e2235ee100b4a8974ca3222b30815eebf148"} Sep 10 13:34:47.373605 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:47.373510 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:47.473138 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:47.473102 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:47.577055 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:47.576997 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-deba50-679c68dccf-g2wgd" podStartSLOduration=1.576979218 podStartE2EDuration="1.576979218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:34:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:34:47.166672599 +0000 UTC m=+997.061390017" watchObservedRunningTime="2026-09-10 13:34:47.576979218 +0000 UTC m=+997.471696636" Sep 10 13:34:47.578130 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:47.578109 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3f02bf87-6769f98cd-bt47t"] Sep 10 13:34:47.581826 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:47.581810 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-3f02bf87-6769f98cd-bt47t" Sep 10 13:34:47.585322 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:47.585300 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-3f02bf87-dockercfg-rjtzb\"" Sep 10 13:34:47.585871 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:47.585853 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-3f02bf87\"" Sep 10 13:34:47.603625 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:47.603593 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3f02bf87-6769f98cd-bt47t"] Sep 10 13:34:47.682672 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:47.680104 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3f02bf87-849b79f84c-qnkp6"] Sep 10 13:34:47.691481 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:47.691453 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3f02bf87-849b79f84c-qnkp6" Sep 10 13:34:47.705527 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:47.705496 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3f02bf87-849b79f84c-qnkp6"] Sep 10 13:34:47.708945 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:47.707275 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2aee2b0-a52a-41c7-a42b-759776c70f37-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-3f02bf87-6769f98cd-bt47t\" (UID: \"b2aee2b0-a52a-41c7-a42b-759776c70f37\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3f02bf87-6769f98cd-bt47t" Sep 10 13:34:47.708945 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:47.707503 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4zc9\" (UniqueName: \"kubernetes.io/projected/b2aee2b0-a52a-41c7-a42b-759776c70f37-kube-api-access-j4zc9\") pod \"payload-pre-processing-e2e-ait-3f02bf87-6769f98cd-bt47t\" (UID: \"b2aee2b0-a52a-41c7-a42b-759776c70f37\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3f02bf87-6769f98cd-bt47t" Sep 10 13:34:47.808092 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:47.808050 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4zc9\" (UniqueName: \"kubernetes.io/projected/b2aee2b0-a52a-41c7-a42b-759776c70f37-kube-api-access-j4zc9\") pod \"payload-pre-processing-e2e-ait-3f02bf87-6769f98cd-bt47t\" (UID: \"b2aee2b0-a52a-41c7-a42b-759776c70f37\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3f02bf87-6769f98cd-bt47t" Sep 10 13:34:47.808304 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:47.808107 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5pqn\" (UniqueName: \"kubernetes.io/projected/e48b4230-cb8a-4712-bbba-a5f0823cfb40-kube-api-access-w5pqn\") pod \"payload-processing-e2e-ait-3f02bf87-849b79f84c-qnkp6\" (UID: \"e48b4230-cb8a-4712-bbba-a5f0823cfb40\") " pod="openshift-ingress/payload-processing-e2e-ait-3f02bf87-849b79f84c-qnkp6" Sep 10 13:34:47.808304 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:47.808130 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e48b4230-cb8a-4712-bbba-a5f0823cfb40-plugins-config-volume\") pod \"payload-processing-e2e-ait-3f02bf87-849b79f84c-qnkp6\" (UID: \"e48b4230-cb8a-4712-bbba-a5f0823cfb40\") " pod="openshift-ingress/payload-processing-e2e-ait-3f02bf87-849b79f84c-qnkp6" Sep 10 13:34:47.808304 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:47.808166 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2aee2b0-a52a-41c7-a42b-759776c70f37-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-3f02bf87-6769f98cd-bt47t\" (UID: \"b2aee2b0-a52a-41c7-a42b-759776c70f37\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3f02bf87-6769f98cd-bt47t" Sep 10 13:34:47.808859 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:47.808841 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2aee2b0-a52a-41c7-a42b-759776c70f37-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-3f02bf87-6769f98cd-bt47t\" (UID: \"b2aee2b0-a52a-41c7-a42b-759776c70f37\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3f02bf87-6769f98cd-bt47t" Sep 10 13:34:47.823314 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:47.823284 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4zc9\" (UniqueName: \"kubernetes.io/projected/b2aee2b0-a52a-41c7-a42b-759776c70f37-kube-api-access-j4zc9\") pod \"payload-pre-processing-e2e-ait-3f02bf87-6769f98cd-bt47t\" (UID: \"b2aee2b0-a52a-41c7-a42b-759776c70f37\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3f02bf87-6769f98cd-bt47t" Sep 10 13:34:47.891600 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:47.891572 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-3f02bf87-6769f98cd-bt47t" Sep 10 13:34:47.909634 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:47.909221 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5pqn\" (UniqueName: \"kubernetes.io/projected/e48b4230-cb8a-4712-bbba-a5f0823cfb40-kube-api-access-w5pqn\") pod \"payload-processing-e2e-ait-3f02bf87-849b79f84c-qnkp6\" (UID: \"e48b4230-cb8a-4712-bbba-a5f0823cfb40\") " pod="openshift-ingress/payload-processing-e2e-ait-3f02bf87-849b79f84c-qnkp6" Sep 10 13:34:47.909634 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:47.909278 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e48b4230-cb8a-4712-bbba-a5f0823cfb40-plugins-config-volume\") pod \"payload-processing-e2e-ait-3f02bf87-849b79f84c-qnkp6\" (UID: \"e48b4230-cb8a-4712-bbba-a5f0823cfb40\") " pod="openshift-ingress/payload-processing-e2e-ait-3f02bf87-849b79f84c-qnkp6" Sep 10 13:34:47.910278 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:47.910251 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e48b4230-cb8a-4712-bbba-a5f0823cfb40-plugins-config-volume\") pod \"payload-processing-e2e-ait-3f02bf87-849b79f84c-qnkp6\" (UID: \"e48b4230-cb8a-4712-bbba-a5f0823cfb40\") " pod="openshift-ingress/payload-processing-e2e-ait-3f02bf87-849b79f84c-qnkp6" Sep 10 13:34:47.926363 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:47.926329 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5pqn\" (UniqueName: \"kubernetes.io/projected/e48b4230-cb8a-4712-bbba-a5f0823cfb40-kube-api-access-w5pqn\") pod \"payload-processing-e2e-ait-3f02bf87-849b79f84c-qnkp6\" (UID: \"e48b4230-cb8a-4712-bbba-a5f0823cfb40\") " pod="openshift-ingress/payload-processing-e2e-ait-3f02bf87-849b79f84c-qnkp6" Sep 10 13:34:48.005811 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:48.005778 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3f02bf87-849b79f84c-qnkp6" Sep 10 13:34:48.095798 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:48.095709 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:48.200094 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:48.200069 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3f02bf87-6769f98cd-bt47t"] Sep 10 13:34:48.201934 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:34:48.201908 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2aee2b0_a52a_41c7_a42b_759776c70f37.slice/crio-218f4b096af6887295849abb4e2563bb73a51d880079fd6217009b34bd756008 WatchSource:0}: Error finding container 218f4b096af6887295849abb4e2563bb73a51d880079fd6217009b34bd756008: Status 404 returned error can't find the container with id 218f4b096af6887295849abb4e2563bb73a51d880079fd6217009b34bd756008 Sep 10 13:34:48.325276 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:48.325244 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3f02bf87-849b79f84c-qnkp6"] Sep 10 13:34:48.325990 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:34:48.325951 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode48b4230_cb8a_4712_bbba_a5f0823cfb40.slice/crio-be0ff1b16fa257723b7314b9c8540011a992ec996b3052b017756ce34a75c019 WatchSource:0}: Error finding container be0ff1b16fa257723b7314b9c8540011a992ec996b3052b017756ce34a75c019: Status 404 returned error can't find the container with id be0ff1b16fa257723b7314b9c8540011a992ec996b3052b017756ce34a75c019 Sep 10 13:34:49.081028 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:49.080990 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:49.128146 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:49.128107 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3f02bf87-849b79f84c-qnkp6" event={"ID":"e48b4230-cb8a-4712-bbba-a5f0823cfb40","Type":"ContainerStarted","Data":"3c9cdb83776cc7d12f6c497e953e04c6b420af736ba38819a5a8ea1ebb54b6da"} Sep 10 13:34:49.128345 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:49.128172 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3f02bf87-849b79f84c-qnkp6" event={"ID":"e48b4230-cb8a-4712-bbba-a5f0823cfb40","Type":"ContainerStarted","Data":"be0ff1b16fa257723b7314b9c8540011a992ec996b3052b017756ce34a75c019"} Sep 10 13:34:49.128345 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:49.128332 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-3f02bf87-849b79f84c-qnkp6" Sep 10 13:34:49.130316 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:49.130287 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-3f02bf87-6769f98cd-bt47t" event={"ID":"b2aee2b0-a52a-41c7-a42b-759776c70f37","Type":"ContainerStarted","Data":"b1c7700b92bcbc53fbd0366cf3b707a360d7167388c1efb47d5dbfe4e83c6c86"} Sep 10 13:34:49.130529 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:49.130513 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-3f02bf87-6769f98cd-bt47t" Sep 10 13:34:49.130652 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:49.130637 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-3f02bf87-6769f98cd-bt47t" event={"ID":"b2aee2b0-a52a-41c7-a42b-759776c70f37","Type":"ContainerStarted","Data":"218f4b096af6887295849abb4e2563bb73a51d880079fd6217009b34bd756008"} Sep 10 13:34:49.156244 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:49.156166 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-3f02bf87-849b79f84c-qnkp6" podStartSLOduration=2.156142846 podStartE2EDuration="2.156142846s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:34:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:34:49.15276276 +0000 UTC m=+999.047480191" watchObservedRunningTime="2026-09-10 13:34:49.156142846 +0000 UTC m=+999.050860265" Sep 10 13:34:49.181901 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:49.181833 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-3f02bf87-6769f98cd-bt47t" podStartSLOduration=2.181812629 podStartE2EDuration="2.181812629s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:34:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:34:49.180282634 +0000 UTC m=+999.075000054" watchObservedRunningTime="2026-09-10 13:34:49.181812629 +0000 UTC m=+999.076530048" Sep 10 13:34:50.071079 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:50.071049 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bc03be-795f8655f4-w88qp" Sep 10 13:34:50.071254 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:50.071134 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-bc03be-58d4c9b479-rqw4t" Sep 10 13:34:50.917577 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:50.917541 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:51.070592 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:51.070550 2560 patch_prober.go:28] interesting pod/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 10 13:34:51.070786 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:51.070623 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" podUID="ef2fd3f9-1e16-4c56-95da-af9379f5ec52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 10 13:34:51.101788 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:51.101753 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:51.201566 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:51.201478 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:51.302012 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:51.301945 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:52.151478 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:52.151443 2560 generic.go:356] "Generic (PLEG): container finished" podID="ef2fd3f9-1e16-4c56-95da-af9379f5ec52" containerID="3d1df39b4bf4f7bb9d2e3a0a57871be6c046aff9432557f63e4d17ee9b09cada" exitCode=0 Sep 10 13:34:52.151891 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:52.151510 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" event={"ID":"ef2fd3f9-1e16-4c56-95da-af9379f5ec52","Type":"ContainerDied","Data":"3d1df39b4bf4f7bb9d2e3a0a57871be6c046aff9432557f63e4d17ee9b09cada"} Sep 10 13:34:52.151936 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:52.151889 2560 scope.go:117] "RemoveContainer" containerID="3d1df39b4bf4f7bb9d2e3a0a57871be6c046aff9432557f63e4d17ee9b09cada" Sep 10 13:34:52.152954 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:52.152928 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:34:52.153014 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:52.152994 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:34:52.153051 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:52.153029 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:34:53.166133 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:53.166088 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" event={"ID":"ef2fd3f9-1e16-4c56-95da-af9379f5ec52","Type":"ContainerStarted","Data":"516f8205c971edd6329ff4a096def270dfe62bbe53cf8ddca8c82a7192a49e8c"} Sep 10 13:34:53.291231 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:53.291188 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:34:53.291486 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:53.291458 2560 patch_prober.go:28] interesting pod/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Sep 10 13:34:53.291546 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:53.291514 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" podUID="ef2fd3f9-1e16-4c56-95da-af9379f5ec52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Sep 10 13:34:54.017308 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:54.017279 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:54.291587 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:54.291501 2560 patch_prober.go:28] interesting pod/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Sep 10 13:34:54.291587 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:54.291552 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" podUID="ef2fd3f9-1e16-4c56-95da-af9379f5ec52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Sep 10 13:34:55.291241 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:55.291199 2560 patch_prober.go:28] interesting pod/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Sep 10 13:34:55.291466 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:55.291264 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" podUID="ef2fd3f9-1e16-4c56-95da-af9379f5ec52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Sep 10 13:34:56.110073 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:56.110042 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:56.291414 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:56.291365 2560 patch_prober.go:28] interesting pod/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Sep 10 13:34:56.291565 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:56.291455 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" podUID="ef2fd3f9-1e16-4c56-95da-af9379f5ec52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Sep 10 13:34:57.291118 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:57.291077 2560 patch_prober.go:28] interesting pod/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Sep 10 13:34:57.291637 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:57.291155 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" podUID="ef2fd3f9-1e16-4c56-95da-af9379f5ec52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Sep 10 13:34:58.121146 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:58.121116 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-deba50-679c68dccf-g2wgd" Sep 10 13:34:58.192048 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:58.191994 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:34:58.294988 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:58.294948 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:34:58.295355 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:58.295246 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:34:58.295930 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:58.295915 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:34:59.972459 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:34:59.972424 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:00.140189 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:00.140160 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-3f02bf87-849b79f84c-qnkp6" Sep 10 13:35:00.314099 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:00.314007 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:00.823193 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:00.823157 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:01.092158 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:01.092065 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:01.147096 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:01.147068 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-3f02bf87-6769f98cd-bt47t" Sep 10 13:35:03.788394 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:03.788355 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:03.907504 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:03.907459 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:06.267864 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:06.267832 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d6159c-564486d59d-scfzx"] Sep 10 13:35:06.271220 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:06.271201 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d6159c-564486d59d-scfzx" Sep 10 13:35:06.276014 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:06.275998 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-d6159c-dockercfg-vjh6b\"" Sep 10 13:35:06.276338 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:06.276325 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-d6159c\"" Sep 10 13:35:06.299117 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:06.299080 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d6159c-564486d59d-scfzx"] Sep 10 13:35:06.325075 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:06.325034 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:06.387948 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:06.387913 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dp4g2\" (UniqueName: \"kubernetes.io/projected/ef9ad2ff-b9fd-4db0-84b6-fa25678dfb3a-kube-api-access-dp4g2\") pod \"payload-pre-processing-e2e-inf-b-d6159c-564486d59d-scfzx\" (UID: \"ef9ad2ff-b9fd-4db0-84b6-fa25678dfb3a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d6159c-564486d59d-scfzx" Sep 10 13:35:06.387948 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:06.387951 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef9ad2ff-b9fd-4db0-84b6-fa25678dfb3a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-d6159c-564486d59d-scfzx\" (UID: \"ef9ad2ff-b9fd-4db0-84b6-fa25678dfb3a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d6159c-564486d59d-scfzx" Sep 10 13:35:06.436213 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:06.436172 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:06.489062 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:06.489026 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dp4g2\" (UniqueName: \"kubernetes.io/projected/ef9ad2ff-b9fd-4db0-84b6-fa25678dfb3a-kube-api-access-dp4g2\") pod \"payload-pre-processing-e2e-inf-b-d6159c-564486d59d-scfzx\" (UID: \"ef9ad2ff-b9fd-4db0-84b6-fa25678dfb3a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d6159c-564486d59d-scfzx" Sep 10 13:35:06.489062 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:06.489062 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef9ad2ff-b9fd-4db0-84b6-fa25678dfb3a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-d6159c-564486d59d-scfzx\" (UID: \"ef9ad2ff-b9fd-4db0-84b6-fa25678dfb3a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d6159c-564486d59d-scfzx" Sep 10 13:35:06.489823 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:06.489803 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef9ad2ff-b9fd-4db0-84b6-fa25678dfb3a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-d6159c-564486d59d-scfzx\" (UID: \"ef9ad2ff-b9fd-4db0-84b6-fa25678dfb3a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d6159c-564486d59d-scfzx" Sep 10 13:35:06.502264 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:06.502231 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dp4g2\" (UniqueName: \"kubernetes.io/projected/ef9ad2ff-b9fd-4db0-84b6-fa25678dfb3a-kube-api-access-dp4g2\") pod \"payload-pre-processing-e2e-inf-b-d6159c-564486d59d-scfzx\" (UID: \"ef9ad2ff-b9fd-4db0-84b6-fa25678dfb3a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d6159c-564486d59d-scfzx" Sep 10 13:35:06.581466 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:06.581376 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d6159c-564486d59d-scfzx" Sep 10 13:35:06.693184 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:06.693150 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:06.873649 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:06.873559 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d6159c-564486d59d-scfzx"] Sep 10 13:35:06.881699 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:35:06.881673 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef9ad2ff_b9fd_4db0_84b6_fa25678dfb3a.slice/crio-90e2bdf4d85f046d95f0f29f241924fc1727f8b8f77880ec0732e323b4834d0d WatchSource:0}: Error finding container 90e2bdf4d85f046d95f0f29f241924fc1727f8b8f77880ec0732e323b4834d0d: Status 404 returned error can't find the container with id 90e2bdf4d85f046d95f0f29f241924fc1727f8b8f77880ec0732e323b4834d0d Sep 10 13:35:07.258744 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:07.258708 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d6159c-564486d59d-scfzx" event={"ID":"ef9ad2ff-b9fd-4db0-84b6-fa25678dfb3a","Type":"ContainerStarted","Data":"e489c5a259442a8b2259d5dc987a2af015d41d320f7220004f443048b8a0f6be"} Sep 10 13:35:07.258939 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:07.258765 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d6159c-564486d59d-scfzx" event={"ID":"ef9ad2ff-b9fd-4db0-84b6-fa25678dfb3a","Type":"ContainerStarted","Data":"90e2bdf4d85f046d95f0f29f241924fc1727f8b8f77880ec0732e323b4834d0d"} Sep 10 13:35:07.258939 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:07.258798 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d6159c-564486d59d-scfzx" Sep 10 13:35:07.288097 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:07.288022 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d6159c-564486d59d-scfzx" podStartSLOduration=1.287996895 podStartE2EDuration="1.287996895s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:35:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:35:07.283620009 +0000 UTC m=+1017.178337426" watchObservedRunningTime="2026-09-10 13:35:07.287996895 +0000 UTC m=+1017.182714317" Sep 10 13:35:07.836029 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:07.835995 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:08.135810 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:08.135727 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:08.509969 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:08.509934 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:08.838617 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:08.838529 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:08.936768 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:08.936738 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:10.008038 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:10.008003 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:15.709920 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:15.709880 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f02bf87-q4l8w"] Sep 10 13:35:15.713653 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:15.713627 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f02bf87-q4l8w" Sep 10 13:35:15.723794 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:15.723773 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 13:35:15.723907 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:15.723840 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pffwn\"" Sep 10 13:35:15.756094 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:15.756061 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f02bf87-q4l8w"] Sep 10 13:35:15.778007 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:15.777970 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4551da67-d377-48a2-9e23-1bdee27676ff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3f02bf87-q4l8w\" (UID: \"4551da67-d377-48a2-9e23-1bdee27676ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f02bf87-q4l8w" Sep 10 13:35:15.879206 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:15.879159 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4551da67-d377-48a2-9e23-1bdee27676ff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3f02bf87-q4l8w\" (UID: \"4551da67-d377-48a2-9e23-1bdee27676ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f02bf87-q4l8w" Sep 10 13:35:15.879852 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:15.879832 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4551da67-d377-48a2-9e23-1bdee27676ff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3f02bf87-q4l8w\" (UID: \"4551da67-d377-48a2-9e23-1bdee27676ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f02bf87-q4l8w" Sep 10 13:35:16.024483 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:16.024362 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f02bf87-q4l8w" Sep 10 13:35:16.301282 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:16.301201 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f02bf87-q4l8w"] Sep 10 13:35:16.304133 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:35:16.304104 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4551da67_d377_48a2_9e23_1bdee27676ff.slice/crio-e1ba63b60cd64a655f4279e17f6592531123dde00983f31ea4239f04e913f24b WatchSource:0}: Error finding container e1ba63b60cd64a655f4279e17f6592531123dde00983f31ea4239f04e913f24b: Status 404 returned error can't find the container with id e1ba63b60cd64a655f4279e17f6592531123dde00983f31ea4239f04e913f24b Sep 10 13:35:16.315134 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:16.315102 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f02bf87-q4l8w" event={"ID":"4551da67-d377-48a2-9e23-1bdee27676ff","Type":"ContainerStarted","Data":"e1ba63b60cd64a655f4279e17f6592531123dde00983f31ea4239f04e913f24b"} Sep 10 13:35:18.267583 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:18.267552 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d6159c-564486d59d-scfzx" Sep 10 13:35:19.336568 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:19.336530 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f02bf87-q4l8w" event={"ID":"4551da67-d377-48a2-9e23-1bdee27676ff","Type":"ContainerStarted","Data":"60a43423c52c206bc4400abd8c00950f5bf32149f80f8618e4b85542795e3eb6"} Sep 10 13:35:19.355502 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:19.355454 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f02bf87-q4l8w" podStartSLOduration=2.054204749 podStartE2EDuration="4.355428211s" totalImagesPullingTime="2.301223462s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:35:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:35:19.353724231 +0000 UTC m=+1029.248441650" watchObservedRunningTime="2026-09-10 13:35:19.355428211 +0000 UTC m=+1029.250145633" Sep 10 13:35:20.344213 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:20.344180 2560 generic.go:356] "Generic (PLEG): container finished" podID="4551da67-d377-48a2-9e23-1bdee27676ff" containerID="60a43423c52c206bc4400abd8c00950f5bf32149f80f8618e4b85542795e3eb6" exitCode=0 Sep 10 13:35:20.344609 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:20.344257 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f02bf87-q4l8w" event={"ID":"4551da67-d377-48a2-9e23-1bdee27676ff","Type":"ContainerDied","Data":"60a43423c52c206bc4400abd8c00950f5bf32149f80f8618e4b85542795e3eb6"} Sep 10 13:35:21.493489 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:21.493382 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f02bf87-q4l8w" Sep 10 13:35:21.637550 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:21.637445 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4551da67-d377-48a2-9e23-1bdee27676ff-maas-api-service-ca\") pod \"4551da67-d377-48a2-9e23-1bdee27676ff\" (UID: \"4551da67-d377-48a2-9e23-1bdee27676ff\") " Sep 10 13:35:21.637839 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:21.637811 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4551da67-d377-48a2-9e23-1bdee27676ff-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4551da67-d377-48a2-9e23-1bdee27676ff" (UID: "4551da67-d377-48a2-9e23-1bdee27676ff"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:35:21.738636 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:21.738598 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4551da67-d377-48a2-9e23-1bdee27676ff-maas-api-service-ca\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:35:22.357051 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:22.357022 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f02bf87-q4l8w" Sep 10 13:35:22.357227 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:22.357053 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f02bf87-q4l8w" event={"ID":"4551da67-d377-48a2-9e23-1bdee27676ff","Type":"ContainerDied","Data":"e1ba63b60cd64a655f4279e17f6592531123dde00983f31ea4239f04e913f24b"} Sep 10 13:35:22.357227 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:22.357104 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e1ba63b60cd64a655f4279e17f6592531123dde00983f31ea4239f04e913f24b" Sep 10 13:35:27.427812 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:27.427759 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:28.526140 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:28.526111 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3f02bf87-849b79f84c-qnkp6"] Sep 10 13:35:28.526557 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:28.526328 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-3f02bf87-849b79f84c-qnkp6" podUID="e48b4230-cb8a-4712-bbba-a5f0823cfb40" containerName="payload-processing" containerID="cri-o://3c9cdb83776cc7d12f6c497e953e04c6b420af736ba38819a5a8ea1ebb54b6da" gracePeriod=30 Sep 10 13:35:28.568729 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:28.568692 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:28.568993 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:28.568972 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3f02bf87-6769f98cd-bt47t"] Sep 10 13:35:28.569169 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:28.569150 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-3f02bf87-6769f98cd-bt47t" podUID="b2aee2b0-a52a-41c7-a42b-759776c70f37" containerName="payload-pre-processing" containerID="cri-o://b1c7700b92bcbc53fbd0366cf3b707a360d7167388c1efb47d5dbfe4e83c6c86" gracePeriod=30 Sep 10 13:35:28.679383 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:28.679351 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:28.816534 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:28.816394 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:28.841679 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:28.841652 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3f02bf87-849b79f84c-qnkp6" Sep 10 13:35:28.855391 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:28.855361 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f02bf87-q4l8w"] Sep 10 13:35:28.859483 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:28.859457 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3f02bf87-q4l8w"] Sep 10 13:35:28.864572 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:28.864552 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-3f02bf87-6769f98cd-bt47t" Sep 10 13:35:28.915221 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:28.915185 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:29.015127 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:29.015085 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2aee2b0-a52a-41c7-a42b-759776c70f37-plugins-config-volume\") pod \"b2aee2b0-a52a-41c7-a42b-759776c70f37\" (UID: \"b2aee2b0-a52a-41c7-a42b-759776c70f37\") " Sep 10 13:35:29.015321 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:29.015169 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j4zc9\" (UniqueName: \"kubernetes.io/projected/b2aee2b0-a52a-41c7-a42b-759776c70f37-kube-api-access-j4zc9\") pod \"b2aee2b0-a52a-41c7-a42b-759776c70f37\" (UID: \"b2aee2b0-a52a-41c7-a42b-759776c70f37\") " Sep 10 13:35:29.015321 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:29.015217 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e48b4230-cb8a-4712-bbba-a5f0823cfb40-plugins-config-volume\") pod \"e48b4230-cb8a-4712-bbba-a5f0823cfb40\" (UID: \"e48b4230-cb8a-4712-bbba-a5f0823cfb40\") " Sep 10 13:35:29.015321 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:29.015269 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w5pqn\" (UniqueName: \"kubernetes.io/projected/e48b4230-cb8a-4712-bbba-a5f0823cfb40-kube-api-access-w5pqn\") pod \"e48b4230-cb8a-4712-bbba-a5f0823cfb40\" (UID: \"e48b4230-cb8a-4712-bbba-a5f0823cfb40\") " Sep 10 13:35:29.015814 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:29.015783 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e48b4230-cb8a-4712-bbba-a5f0823cfb40-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e48b4230-cb8a-4712-bbba-a5f0823cfb40" (UID: "e48b4230-cb8a-4712-bbba-a5f0823cfb40"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:35:29.016136 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:29.016112 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2aee2b0-a52a-41c7-a42b-759776c70f37-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b2aee2b0-a52a-41c7-a42b-759776c70f37" (UID: "b2aee2b0-a52a-41c7-a42b-759776c70f37"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:35:29.018269 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:29.018246 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2aee2b0-a52a-41c7-a42b-759776c70f37-kube-api-access-j4zc9" (OuterVolumeSpecName: "kube-api-access-j4zc9") pod "b2aee2b0-a52a-41c7-a42b-759776c70f37" (UID: "b2aee2b0-a52a-41c7-a42b-759776c70f37"). InnerVolumeSpecName "kube-api-access-j4zc9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:35:29.018486 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:29.018457 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e48b4230-cb8a-4712-bbba-a5f0823cfb40-kube-api-access-w5pqn" (OuterVolumeSpecName: "kube-api-access-w5pqn") pod "e48b4230-cb8a-4712-bbba-a5f0823cfb40" (UID: "e48b4230-cb8a-4712-bbba-a5f0823cfb40"). InnerVolumeSpecName "kube-api-access-w5pqn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:35:29.116907 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:29.116821 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2aee2b0-a52a-41c7-a42b-759776c70f37-plugins-config-volume\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:35:29.116907 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:29.116853 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4zc9\" (UniqueName: \"kubernetes.io/projected/b2aee2b0-a52a-41c7-a42b-759776c70f37-kube-api-access-j4zc9\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:35:29.116907 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:29.116863 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e48b4230-cb8a-4712-bbba-a5f0823cfb40-plugins-config-volume\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:35:29.116907 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:29.116872 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w5pqn\" (UniqueName: \"kubernetes.io/projected/e48b4230-cb8a-4712-bbba-a5f0823cfb40-kube-api-access-w5pqn\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:35:29.119634 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:29.119604 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:29.400841 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:29.400747 2560 generic.go:356] "Generic (PLEG): container finished" podID="e48b4230-cb8a-4712-bbba-a5f0823cfb40" containerID="3c9cdb83776cc7d12f6c497e953e04c6b420af736ba38819a5a8ea1ebb54b6da" exitCode=0 Sep 10 13:35:29.400841 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:29.400796 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3f02bf87-849b79f84c-qnkp6" Sep 10 13:35:29.400841 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:29.400832 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3f02bf87-849b79f84c-qnkp6" event={"ID":"e48b4230-cb8a-4712-bbba-a5f0823cfb40","Type":"ContainerDied","Data":"3c9cdb83776cc7d12f6c497e953e04c6b420af736ba38819a5a8ea1ebb54b6da"} Sep 10 13:35:29.401084 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:29.400879 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3f02bf87-849b79f84c-qnkp6" event={"ID":"e48b4230-cb8a-4712-bbba-a5f0823cfb40","Type":"ContainerDied","Data":"be0ff1b16fa257723b7314b9c8540011a992ec996b3052b017756ce34a75c019"} Sep 10 13:35:29.401084 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:29.400902 2560 scope.go:117] "RemoveContainer" containerID="3c9cdb83776cc7d12f6c497e953e04c6b420af736ba38819a5a8ea1ebb54b6da" Sep 10 13:35:29.402338 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:29.402316 2560 generic.go:356] "Generic (PLEG): container finished" podID="b2aee2b0-a52a-41c7-a42b-759776c70f37" containerID="b1c7700b92bcbc53fbd0366cf3b707a360d7167388c1efb47d5dbfe4e83c6c86" exitCode=0 Sep 10 13:35:29.402446 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:29.402351 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-3f02bf87-6769f98cd-bt47t" Sep 10 13:35:29.402446 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:29.402390 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-3f02bf87-6769f98cd-bt47t" event={"ID":"b2aee2b0-a52a-41c7-a42b-759776c70f37","Type":"ContainerDied","Data":"b1c7700b92bcbc53fbd0366cf3b707a360d7167388c1efb47d5dbfe4e83c6c86"} Sep 10 13:35:29.402564 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:29.402451 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-3f02bf87-6769f98cd-bt47t" event={"ID":"b2aee2b0-a52a-41c7-a42b-759776c70f37","Type":"ContainerDied","Data":"218f4b096af6887295849abb4e2563bb73a51d880079fd6217009b34bd756008"} Sep 10 13:35:29.411897 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:29.411875 2560 scope.go:117] "RemoveContainer" containerID="3c9cdb83776cc7d12f6c497e953e04c6b420af736ba38819a5a8ea1ebb54b6da" Sep 10 13:35:29.412474 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:35:29.412240 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c9cdb83776cc7d12f6c497e953e04c6b420af736ba38819a5a8ea1ebb54b6da\": container with ID starting with 3c9cdb83776cc7d12f6c497e953e04c6b420af736ba38819a5a8ea1ebb54b6da not found: ID does not exist" containerID="3c9cdb83776cc7d12f6c497e953e04c6b420af736ba38819a5a8ea1ebb54b6da" Sep 10 13:35:29.412474 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:29.412284 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c9cdb83776cc7d12f6c497e953e04c6b420af736ba38819a5a8ea1ebb54b6da"} err="failed to get container status \"3c9cdb83776cc7d12f6c497e953e04c6b420af736ba38819a5a8ea1ebb54b6da\": rpc error: code = NotFound desc = could not find container \"3c9cdb83776cc7d12f6c497e953e04c6b420af736ba38819a5a8ea1ebb54b6da\": container with ID starting with 3c9cdb83776cc7d12f6c497e953e04c6b420af736ba38819a5a8ea1ebb54b6da not found: ID does not exist" Sep 10 13:35:29.412474 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:29.412303 2560 scope.go:117] "RemoveContainer" containerID="b1c7700b92bcbc53fbd0366cf3b707a360d7167388c1efb47d5dbfe4e83c6c86" Sep 10 13:35:29.421477 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:29.421460 2560 scope.go:117] "RemoveContainer" containerID="b1c7700b92bcbc53fbd0366cf3b707a360d7167388c1efb47d5dbfe4e83c6c86" Sep 10 13:35:29.421754 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:35:29.421737 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1c7700b92bcbc53fbd0366cf3b707a360d7167388c1efb47d5dbfe4e83c6c86\": container with ID starting with b1c7700b92bcbc53fbd0366cf3b707a360d7167388c1efb47d5dbfe4e83c6c86 not found: ID does not exist" containerID="b1c7700b92bcbc53fbd0366cf3b707a360d7167388c1efb47d5dbfe4e83c6c86" Sep 10 13:35:29.421805 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:29.421760 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1c7700b92bcbc53fbd0366cf3b707a360d7167388c1efb47d5dbfe4e83c6c86"} err="failed to get container status \"b1c7700b92bcbc53fbd0366cf3b707a360d7167388c1efb47d5dbfe4e83c6c86\": rpc error: code = NotFound desc = could not find container \"b1c7700b92bcbc53fbd0366cf3b707a360d7167388c1efb47d5dbfe4e83c6c86\": container with ID starting with b1c7700b92bcbc53fbd0366cf3b707a360d7167388c1efb47d5dbfe4e83c6c86 not found: ID does not exist" Sep 10 13:35:29.436781 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:29.436751 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3f02bf87-6769f98cd-bt47t"] Sep 10 13:35:29.439705 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:29.439682 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3f02bf87-6769f98cd-bt47t"] Sep 10 13:35:29.456745 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:29.456716 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3f02bf87-849b79f84c-qnkp6"] Sep 10 13:35:29.460770 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:29.460740 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3f02bf87-849b79f84c-qnkp6"] Sep 10 13:35:29.605991 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:29.605958 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:29.730366 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:29.730336 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:29.936595 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:29.936562 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:30.365621 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.365589 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv"] Sep 10 13:35:30.366055 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.366042 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4551da67-d377-48a2-9e23-1bdee27676ff" containerName="revoke-keys" Sep 10 13:35:30.366109 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.366057 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4551da67-d377-48a2-9e23-1bdee27676ff" containerName="revoke-keys" Sep 10 13:35:30.366109 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.366080 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2aee2b0-a52a-41c7-a42b-759776c70f37" containerName="payload-pre-processing" Sep 10 13:35:30.366109 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.366088 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2aee2b0-a52a-41c7-a42b-759776c70f37" containerName="payload-pre-processing" Sep 10 13:35:30.366109 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.366108 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e48b4230-cb8a-4712-bbba-a5f0823cfb40" containerName="payload-processing" Sep 10 13:35:30.366228 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.366115 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e48b4230-cb8a-4712-bbba-a5f0823cfb40" containerName="payload-processing" Sep 10 13:35:30.366228 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.366196 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2aee2b0-a52a-41c7-a42b-759776c70f37" containerName="payload-pre-processing" Sep 10 13:35:30.366228 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.366207 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4551da67-d377-48a2-9e23-1bdee27676ff" containerName="revoke-keys" Sep 10 13:35:30.366228 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.366216 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e48b4230-cb8a-4712-bbba-a5f0823cfb40" containerName="payload-processing" Sep 10 13:35:30.370979 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.370960 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:35:30.376386 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.376361 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-4907abfd-gw-openshift-default-dockercfg-gb8np\"" Sep 10 13:35:30.398077 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.398050 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:30.398824 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.398800 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv"] Sep 10 13:35:30.532575 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.532533 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-credential-socket\") pod \"e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv\" (UID: \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\") " pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:35:30.532575 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.532572 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-workload-socket\") pod \"e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv\" (UID: \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\") " pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:35:30.532847 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.532635 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75s2c\" (UniqueName: \"kubernetes.io/projected/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-kube-api-access-75s2c\") pod \"e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv\" (UID: \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\") " pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:35:30.532847 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.532769 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-istio-token\") pod \"e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv\" (UID: \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\") " pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:35:30.532847 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.532839 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-istio-envoy\") pod \"e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv\" (UID: \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\") " pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:35:30.533015 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.532879 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-istio-data\") pod \"e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv\" (UID: \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\") " pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:35:30.533015 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.532940 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-istio-podinfo\") pod \"e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv\" (UID: \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\") " pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:35:30.533015 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.533005 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-workload-certs\") pod \"e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv\" (UID: \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\") " pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:35:30.533129 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.533033 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-istiod-ca-cert\") pod \"e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv\" (UID: \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\") " pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:35:30.564411 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.564372 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4551da67-d377-48a2-9e23-1bdee27676ff" path="/var/lib/kubelet/pods/4551da67-d377-48a2-9e23-1bdee27676ff/volumes" Sep 10 13:35:30.564949 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.564932 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2aee2b0-a52a-41c7-a42b-759776c70f37" path="/var/lib/kubelet/pods/b2aee2b0-a52a-41c7-a42b-759776c70f37/volumes" Sep 10 13:35:30.565384 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.565370 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e48b4230-cb8a-4712-bbba-a5f0823cfb40" path="/var/lib/kubelet/pods/e48b4230-cb8a-4712-bbba-a5f0823cfb40/volumes" Sep 10 13:35:30.634300 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.634210 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-workload-certs\") pod \"e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv\" (UID: \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\") " pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:35:30.634300 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.634259 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-istiod-ca-cert\") pod \"e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv\" (UID: \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\") " pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:35:30.634821 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.634309 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-credential-socket\") pod \"e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv\" (UID: \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\") " pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:35:30.634821 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.634335 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-workload-socket\") pod \"e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv\" (UID: \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\") " pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:35:30.634821 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.634372 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75s2c\" (UniqueName: \"kubernetes.io/projected/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-kube-api-access-75s2c\") pod \"e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv\" (UID: \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\") " pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:35:30.634821 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.634447 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-istio-token\") pod \"e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv\" (UID: \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\") " pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:35:30.634821 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.634515 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-istio-envoy\") pod \"e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv\" (UID: \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\") " pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:35:30.634821 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.634582 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-istio-data\") pod \"e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv\" (UID: \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\") " pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:35:30.634821 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.634652 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-istio-podinfo\") pod \"e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv\" (UID: \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\") " pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:35:30.634821 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.634662 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-workload-certs\") pod \"e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv\" (UID: \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\") " pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:35:30.634821 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.634704 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-credential-socket\") pod \"e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv\" (UID: \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\") " pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:35:30.635217 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.635046 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-workload-socket\") pod \"e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv\" (UID: \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\") " pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:35:30.635217 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.635087 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-istio-data\") pod \"e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv\" (UID: \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\") " pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:35:30.635217 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.635143 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-istiod-ca-cert\") pod \"e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv\" (UID: \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\") " pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:35:30.637120 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.637100 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-istio-podinfo\") pod \"e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv\" (UID: \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\") " pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:35:30.637227 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.637182 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-istio-envoy\") pod \"e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv\" (UID: \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\") " pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:35:30.649543 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.649516 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-istio-token\") pod \"e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv\" (UID: \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\") " pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:35:30.649673 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.649654 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75s2c\" (UniqueName: \"kubernetes.io/projected/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-kube-api-access-75s2c\") pod \"e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv\" (UID: \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\") " pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:35:30.681200 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.681164 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:35:30.976252 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.976225 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv"] Sep 10 13:35:30.976777 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:35:30.976751 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1017b5b4_ae1f_4925_b29b_3a87740fc0a5.slice/crio-3bb22c8fe588c1ede9700ee86be2801738018c4ef7ea9849c22a0115787a1b3a WatchSource:0}: Error finding container 3bb22c8fe588c1ede9700ee86be2801738018c4ef7ea9849c22a0115787a1b3a: Status 404 returned error can't find the container with id 3bb22c8fe588c1ede9700ee86be2801738018c4ef7ea9849c22a0115787a1b3a Sep 10 13:35:30.979528 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.979495 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:35:30.979627 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.979576 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:35:30.979627 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:30.979622 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:35:31.339735 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:31.339646 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:31.421952 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:31.421911 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" event={"ID":"1017b5b4-ae1f-4925-b29b-3a87740fc0a5","Type":"ContainerStarted","Data":"1d4bdd141cfbe33127bfb8b3c07ea11de257d40e78e1ab73404c7fe8896be614"} Sep 10 13:35:31.422145 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:31.421965 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" event={"ID":"1017b5b4-ae1f-4925-b29b-3a87740fc0a5","Type":"ContainerStarted","Data":"3bb22c8fe588c1ede9700ee86be2801738018c4ef7ea9849c22a0115787a1b3a"} Sep 10 13:35:31.479767 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:31.479699 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" podStartSLOduration=1.47967444 podStartE2EDuration="1.47967444s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:35:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:35:31.478503373 +0000 UTC m=+1041.373220791" watchObservedRunningTime="2026-09-10 13:35:31.47967444 +0000 UTC m=+1041.374391901" Sep 10 13:35:31.682031 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:31.681998 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:35:32.613076 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:32.613036 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:32.687676 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:32.687636 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:35:33.131873 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:33.131837 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:33.237366 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:33.237332 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:33.435800 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:33.435774 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:35:33.436962 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:33.436936 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:35:33.450139 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:33.450110 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:33.553663 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:33.553630 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:33.890618 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:33.890502 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:34.095372 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:34.095335 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:34.198897 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:34.198864 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:34.314678 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:34.314637 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-4907abfd-55cf7d755d-cwx4p"] Sep 10 13:35:34.323448 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:34.323376 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-4907abfd-55cf7d755d-cwx4p" Sep 10 13:35:34.327192 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:34.327156 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-4907abfd-dockercfg-f9qnq\"" Sep 10 13:35:34.327340 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:34.327230 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-4907abfd\"" Sep 10 13:35:34.333112 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:34.333083 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-4907abfd-55cf7d755d-cwx4p"] Sep 10 13:35:34.373663 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:34.373618 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-4907abfd-5f6fbc6cbb-6tq2q"] Sep 10 13:35:34.378324 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:34.378295 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-4907abfd-5f6fbc6cbb-6tq2q" Sep 10 13:35:34.396488 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:34.396452 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-4907abfd-5f6fbc6cbb-6tq2q"] Sep 10 13:35:34.476710 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:34.476611 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3cd6f5c-2f78-4a22-8a4b-a6911ac25412-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-4907abfd-55cf7d755d-cwx4p\" (UID: \"a3cd6f5c-2f78-4a22-8a4b-a6911ac25412\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-4907abfd-55cf7d755d-cwx4p" Sep 10 13:35:34.476710 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:34.476650 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czg5l\" (UniqueName: \"kubernetes.io/projected/774cd87d-fc9a-452d-85c5-b758b09eb50d-kube-api-access-czg5l\") pod \"payload-processing-e2e-migrate-4907abfd-5f6fbc6cbb-6tq2q\" (UID: \"774cd87d-fc9a-452d-85c5-b758b09eb50d\") " pod="openshift-ingress/payload-processing-e2e-migrate-4907abfd-5f6fbc6cbb-6tq2q" Sep 10 13:35:34.476925 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:34.476726 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqchg\" (UniqueName: \"kubernetes.io/projected/a3cd6f5c-2f78-4a22-8a4b-a6911ac25412-kube-api-access-cqchg\") pod \"payload-pre-processing-e2e-migrate-4907abfd-55cf7d755d-cwx4p\" (UID: \"a3cd6f5c-2f78-4a22-8a4b-a6911ac25412\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-4907abfd-55cf7d755d-cwx4p" Sep 10 13:35:34.476925 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:34.476760 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/774cd87d-fc9a-452d-85c5-b758b09eb50d-plugins-config-volume\") pod \"payload-processing-e2e-migrate-4907abfd-5f6fbc6cbb-6tq2q\" (UID: \"774cd87d-fc9a-452d-85c5-b758b09eb50d\") " pod="openshift-ingress/payload-processing-e2e-migrate-4907abfd-5f6fbc6cbb-6tq2q" Sep 10 13:35:34.578113 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:34.578077 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3cd6f5c-2f78-4a22-8a4b-a6911ac25412-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-4907abfd-55cf7d755d-cwx4p\" (UID: \"a3cd6f5c-2f78-4a22-8a4b-a6911ac25412\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-4907abfd-55cf7d755d-cwx4p" Sep 10 13:35:34.578113 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:34.578115 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czg5l\" (UniqueName: \"kubernetes.io/projected/774cd87d-fc9a-452d-85c5-b758b09eb50d-kube-api-access-czg5l\") pod \"payload-processing-e2e-migrate-4907abfd-5f6fbc6cbb-6tq2q\" (UID: \"774cd87d-fc9a-452d-85c5-b758b09eb50d\") " pod="openshift-ingress/payload-processing-e2e-migrate-4907abfd-5f6fbc6cbb-6tq2q" Sep 10 13:35:34.578370 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:34.578156 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqchg\" (UniqueName: \"kubernetes.io/projected/a3cd6f5c-2f78-4a22-8a4b-a6911ac25412-kube-api-access-cqchg\") pod \"payload-pre-processing-e2e-migrate-4907abfd-55cf7d755d-cwx4p\" (UID: \"a3cd6f5c-2f78-4a22-8a4b-a6911ac25412\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-4907abfd-55cf7d755d-cwx4p" Sep 10 13:35:34.578370 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:34.578180 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/774cd87d-fc9a-452d-85c5-b758b09eb50d-plugins-config-volume\") pod \"payload-processing-e2e-migrate-4907abfd-5f6fbc6cbb-6tq2q\" (UID: \"774cd87d-fc9a-452d-85c5-b758b09eb50d\") " pod="openshift-ingress/payload-processing-e2e-migrate-4907abfd-5f6fbc6cbb-6tq2q" Sep 10 13:35:34.579084 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:34.579059 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/774cd87d-fc9a-452d-85c5-b758b09eb50d-plugins-config-volume\") pod \"payload-processing-e2e-migrate-4907abfd-5f6fbc6cbb-6tq2q\" (UID: \"774cd87d-fc9a-452d-85c5-b758b09eb50d\") " pod="openshift-ingress/payload-processing-e2e-migrate-4907abfd-5f6fbc6cbb-6tq2q" Sep 10 13:35:34.579145 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:34.579058 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3cd6f5c-2f78-4a22-8a4b-a6911ac25412-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-4907abfd-55cf7d755d-cwx4p\" (UID: \"a3cd6f5c-2f78-4a22-8a4b-a6911ac25412\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-4907abfd-55cf7d755d-cwx4p" Sep 10 13:35:34.595261 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:34.595225 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqchg\" (UniqueName: \"kubernetes.io/projected/a3cd6f5c-2f78-4a22-8a4b-a6911ac25412-kube-api-access-cqchg\") pod \"payload-pre-processing-e2e-migrate-4907abfd-55cf7d755d-cwx4p\" (UID: \"a3cd6f5c-2f78-4a22-8a4b-a6911ac25412\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-4907abfd-55cf7d755d-cwx4p" Sep 10 13:35:34.595392 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:34.595320 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czg5l\" (UniqueName: \"kubernetes.io/projected/774cd87d-fc9a-452d-85c5-b758b09eb50d-kube-api-access-czg5l\") pod \"payload-processing-e2e-migrate-4907abfd-5f6fbc6cbb-6tq2q\" (UID: \"774cd87d-fc9a-452d-85c5-b758b09eb50d\") " pod="openshift-ingress/payload-processing-e2e-migrate-4907abfd-5f6fbc6cbb-6tq2q" Sep 10 13:35:34.636805 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:34.636768 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-4907abfd-55cf7d755d-cwx4p" Sep 10 13:35:34.693604 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:34.693571 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-4907abfd-5f6fbc6cbb-6tq2q" Sep 10 13:35:34.936156 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:35:34.936129 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3cd6f5c_2f78_4a22_8a4b_a6911ac25412.slice/crio-ab310dbf80946a38622a3654d51de0678d02c3a8c0128c5a3bb77ab3df7d8fb4 WatchSource:0}: Error finding container ab310dbf80946a38622a3654d51de0678d02c3a8c0128c5a3bb77ab3df7d8fb4: Status 404 returned error can't find the container with id ab310dbf80946a38622a3654d51de0678d02c3a8c0128c5a3bb77ab3df7d8fb4 Sep 10 13:35:34.936505 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:34.936176 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-4907abfd-55cf7d755d-cwx4p"] Sep 10 13:35:34.991151 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:35:34.991120 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod774cd87d_fc9a_452d_85c5_b758b09eb50d.slice/crio-c0fec5c7e07bcf1a15ea809cc9fd4ee4d53a6189cbf24cef70febafa8566cc48 WatchSource:0}: Error finding container c0fec5c7e07bcf1a15ea809cc9fd4ee4d53a6189cbf24cef70febafa8566cc48: Status 404 returned error can't find the container with id c0fec5c7e07bcf1a15ea809cc9fd4ee4d53a6189cbf24cef70febafa8566cc48 Sep 10 13:35:34.991563 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:34.991542 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-4907abfd-5f6fbc6cbb-6tq2q"] Sep 10 13:35:35.450784 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:35.450751 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-4907abfd-5f6fbc6cbb-6tq2q" event={"ID":"774cd87d-fc9a-452d-85c5-b758b09eb50d","Type":"ContainerStarted","Data":"7dbf40afd01142bdaa0a34684d7faa445be59e751a8236e064162fb0d00e9551"} Sep 10 13:35:35.450997 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:35.450814 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-4907abfd-5f6fbc6cbb-6tq2q" event={"ID":"774cd87d-fc9a-452d-85c5-b758b09eb50d","Type":"ContainerStarted","Data":"c0fec5c7e07bcf1a15ea809cc9fd4ee4d53a6189cbf24cef70febafa8566cc48"} Sep 10 13:35:35.450997 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:35.450845 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-4907abfd-5f6fbc6cbb-6tq2q" Sep 10 13:35:35.452563 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:35.452534 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-4907abfd-55cf7d755d-cwx4p" event={"ID":"a3cd6f5c-2f78-4a22-8a4b-a6911ac25412","Type":"ContainerStarted","Data":"b78aa4bb09c1afd7a4863b255bb9c8e84f218a55dccfbdfdbbf3011cc471ad1d"} Sep 10 13:35:35.452678 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:35.452585 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-4907abfd-55cf7d755d-cwx4p" event={"ID":"a3cd6f5c-2f78-4a22-8a4b-a6911ac25412","Type":"ContainerStarted","Data":"ab310dbf80946a38622a3654d51de0678d02c3a8c0128c5a3bb77ab3df7d8fb4"} Sep 10 13:35:35.452820 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:35.452802 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-4907abfd-55cf7d755d-cwx4p" Sep 10 13:35:35.477152 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:35.477097 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-4907abfd-5f6fbc6cbb-6tq2q" podStartSLOduration=1.477077131 podStartE2EDuration="1.477077131s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:35:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:35:35.473030854 +0000 UTC m=+1045.367748275" watchObservedRunningTime="2026-09-10 13:35:35.477077131 +0000 UTC m=+1045.371794601" Sep 10 13:35:35.510080 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:35.510012 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-4907abfd-55cf7d755d-cwx4p" podStartSLOduration=1.509996615 podStartE2EDuration="1.509996615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:35:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:35:35.506696697 +0000 UTC m=+1045.401414124" watchObservedRunningTime="2026-09-10 13:35:35.509996615 +0000 UTC m=+1045.404714032" Sep 10 13:35:43.930979 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:43.930939 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:44.087247 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:44.087206 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:44.288316 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:44.288223 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:46.460608 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:46.460579 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-4907abfd-55cf7d755d-cwx4p" Sep 10 13:35:46.460990 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:46.460754 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-4907abfd-5f6fbc6cbb-6tq2q" Sep 10 13:35:50.888997 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:50.888955 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:51.007552 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:51.007514 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:51.135825 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:51.135779 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:51.364787 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:51.364751 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:53.280325 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:53.280279 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:56.589808 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:56.589724 2560 generic.go:356] "Generic (PLEG): container finished" podID="1017b5b4-ae1f-4925-b29b-3a87740fc0a5" containerID="1d4bdd141cfbe33127bfb8b3c07ea11de257d40e78e1ab73404c7fe8896be614" exitCode=0 Sep 10 13:35:56.590207 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:56.589805 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" event={"ID":"1017b5b4-ae1f-4925-b29b-3a87740fc0a5","Type":"ContainerDied","Data":"1d4bdd141cfbe33127bfb8b3c07ea11de257d40e78e1ab73404c7fe8896be614"} Sep 10 13:35:56.590263 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:56.590240 2560 scope.go:117] "RemoveContainer" containerID="1d4bdd141cfbe33127bfb8b3c07ea11de257d40e78e1ab73404c7fe8896be614" Sep 10 13:35:56.591430 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:56.591372 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:35:56.591522 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:56.591466 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:35:56.591522 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:56.591504 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:35:57.597364 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:57.597330 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" event={"ID":"1017b5b4-ae1f-4925-b29b-3a87740fc0a5","Type":"ContainerStarted","Data":"2670009207e8450e184a98384a11a6244883f3553bee61ae4b0f32d2c24af036"} Sep 10 13:35:57.682198 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:57.682159 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:35:57.682449 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:57.682395 2560 patch_prober.go:28] interesting pod/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Sep 10 13:35:57.682504 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:57.682476 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" podUID="1017b5b4-ae1f-4925-b29b-3a87740fc0a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Sep 10 13:35:57.849600 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:57.849512 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:57.952132 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:57.952096 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:35:58.682484 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:58.682450 2560 patch_prober.go:28] interesting pod/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Sep 10 13:35:58.682884 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:58.682505 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" podUID="1017b5b4-ae1f-4925-b29b-3a87740fc0a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Sep 10 13:35:59.682661 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:59.682622 2560 patch_prober.go:28] interesting pod/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Sep 10 13:35:59.683081 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:35:59.682683 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" podUID="1017b5b4-ae1f-4925-b29b-3a87740fc0a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Sep 10 13:36:00.616478 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:00.616433 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:00.682090 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:00.682052 2560 patch_prober.go:28] interesting pod/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Sep 10 13:36:00.682273 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:00.682117 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" podUID="1017b5b4-ae1f-4925-b29b-3a87740fc0a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Sep 10 13:36:01.682239 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:01.682200 2560 patch_prober.go:28] interesting pod/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Sep 10 13:36:01.682638 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:01.682276 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" podUID="1017b5b4-ae1f-4925-b29b-3a87740fc0a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Sep 10 13:36:02.682581 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:02.682548 2560 patch_prober.go:28] interesting pod/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Sep 10 13:36:02.682972 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:02.682617 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" podUID="1017b5b4-ae1f-4925-b29b-3a87740fc0a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Sep 10 13:36:03.436441 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:03.436377 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:36:03.682514 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:03.682479 2560 patch_prober.go:28] interesting pod/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Sep 10 13:36:03.682669 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:03.682532 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" podUID="1017b5b4-ae1f-4925-b29b-3a87740fc0a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Sep 10 13:36:03.989009 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:03.988978 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:04.686256 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:04.686223 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:36:04.687351 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:04.687329 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:36:05.480046 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:05.480005 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:10.317806 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:10.317773 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:10.414951 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:10.414918 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:14.727151 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:14.727113 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:14.777869 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:14.777822 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-4907abfd-55cf7d755d-cwx4p"] Sep 10 13:36:14.778155 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:14.778031 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-4907abfd-55cf7d755d-cwx4p" podUID="a3cd6f5c-2f78-4a22-8a4b-a6911ac25412" containerName="payload-pre-processing" containerID="cri-o://b78aa4bb09c1afd7a4863b255bb9c8e84f218a55dccfbdfdbbf3011cc471ad1d" gracePeriod=30 Sep 10 13:36:14.780807 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:14.780781 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-4907abfd-5f6fbc6cbb-6tq2q"] Sep 10 13:36:14.780976 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:14.780955 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-4907abfd-5f6fbc6cbb-6tq2q" podUID="774cd87d-fc9a-452d-85c5-b758b09eb50d" containerName="payload-processing" containerID="cri-o://7dbf40afd01142bdaa0a34684d7faa445be59e751a8236e064162fb0d00e9551" gracePeriod=30 Sep 10 13:36:14.841918 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:14.841884 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:14.969488 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:14.969458 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:15.038554 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.038518 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-deba50-679c68dccf-g2wgd"] Sep 10 13:36:15.039055 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.039020 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-deba50-679c68dccf-g2wgd" podUID="225727ab-44d8-4879-bee7-70a5e2668cfb" containerName="payload-processing" containerID="cri-o://e45feffd4ff81c0a6e0a8a4c3328339cb90c8daccd07f0d4bfe72f7696864dd4" gracePeriod=30 Sep 10 13:36:15.075217 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.075179 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:15.169037 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.169012 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-4907abfd-55cf7d755d-cwx4p" Sep 10 13:36:15.173769 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.173747 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-4907abfd-5f6fbc6cbb-6tq2q" Sep 10 13:36:15.180830 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.180798 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3cd6f5c-2f78-4a22-8a4b-a6911ac25412-plugins-config-volume\") pod \"a3cd6f5c-2f78-4a22-8a4b-a6911ac25412\" (UID: \"a3cd6f5c-2f78-4a22-8a4b-a6911ac25412\") " Sep 10 13:36:15.180957 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.180866 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cqchg\" (UniqueName: \"kubernetes.io/projected/a3cd6f5c-2f78-4a22-8a4b-a6911ac25412-kube-api-access-cqchg\") pod \"a3cd6f5c-2f78-4a22-8a4b-a6911ac25412\" (UID: \"a3cd6f5c-2f78-4a22-8a4b-a6911ac25412\") " Sep 10 13:36:15.180957 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.180904 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-czg5l\" (UniqueName: \"kubernetes.io/projected/774cd87d-fc9a-452d-85c5-b758b09eb50d-kube-api-access-czg5l\") pod \"774cd87d-fc9a-452d-85c5-b758b09eb50d\" (UID: \"774cd87d-fc9a-452d-85c5-b758b09eb50d\") " Sep 10 13:36:15.181068 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.180977 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/774cd87d-fc9a-452d-85c5-b758b09eb50d-plugins-config-volume\") pod \"774cd87d-fc9a-452d-85c5-b758b09eb50d\" (UID: \"774cd87d-fc9a-452d-85c5-b758b09eb50d\") " Sep 10 13:36:15.181219 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.181193 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3cd6f5c-2f78-4a22-8a4b-a6911ac25412-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a3cd6f5c-2f78-4a22-8a4b-a6911ac25412" (UID: "a3cd6f5c-2f78-4a22-8a4b-a6911ac25412"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:36:15.181505 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.181472 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/774cd87d-fc9a-452d-85c5-b758b09eb50d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "774cd87d-fc9a-452d-85c5-b758b09eb50d" (UID: "774cd87d-fc9a-452d-85c5-b758b09eb50d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:36:15.181505 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.181501 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3cd6f5c-2f78-4a22-8a4b-a6911ac25412-plugins-config-volume\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:36:15.184451 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.184421 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3cd6f5c-2f78-4a22-8a4b-a6911ac25412-kube-api-access-cqchg" (OuterVolumeSpecName: "kube-api-access-cqchg") pod "a3cd6f5c-2f78-4a22-8a4b-a6911ac25412" (UID: "a3cd6f5c-2f78-4a22-8a4b-a6911ac25412"). InnerVolumeSpecName "kube-api-access-cqchg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:36:15.185379 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.185345 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/774cd87d-fc9a-452d-85c5-b758b09eb50d-kube-api-access-czg5l" (OuterVolumeSpecName: "kube-api-access-czg5l") pod "774cd87d-fc9a-452d-85c5-b758b09eb50d" (UID: "774cd87d-fc9a-452d-85c5-b758b09eb50d"). InnerVolumeSpecName "kube-api-access-czg5l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:36:15.239130 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.239095 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv"] Sep 10 13:36:15.239391 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.239344 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" podUID="1017b5b4-ae1f-4925-b29b-3a87740fc0a5" containerName="istio-proxy" containerID="cri-o://2670009207e8450e184a98384a11a6244883f3553bee61ae4b0f32d2c24af036" gracePeriod=30 Sep 10 13:36:15.282329 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.282247 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/774cd87d-fc9a-452d-85c5-b758b09eb50d-plugins-config-volume\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:36:15.282329 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.282283 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqchg\" (UniqueName: \"kubernetes.io/projected/a3cd6f5c-2f78-4a22-8a4b-a6911ac25412-kube-api-access-cqchg\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:36:15.282329 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.282299 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czg5l\" (UniqueName: \"kubernetes.io/projected/774cd87d-fc9a-452d-85c5-b758b09eb50d-kube-api-access-czg5l\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:36:15.320643 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.320606 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-deba50-679c68dccf-g2wgd" Sep 10 13:36:15.383729 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.383694 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b9nzm\" (UniqueName: \"kubernetes.io/projected/225727ab-44d8-4879-bee7-70a5e2668cfb-kube-api-access-b9nzm\") pod \"225727ab-44d8-4879-bee7-70a5e2668cfb\" (UID: \"225727ab-44d8-4879-bee7-70a5e2668cfb\") " Sep 10 13:36:15.383928 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.383750 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/225727ab-44d8-4879-bee7-70a5e2668cfb-plugins-config-volume\") pod \"225727ab-44d8-4879-bee7-70a5e2668cfb\" (UID: \"225727ab-44d8-4879-bee7-70a5e2668cfb\") " Sep 10 13:36:15.384627 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.384593 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/225727ab-44d8-4879-bee7-70a5e2668cfb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "225727ab-44d8-4879-bee7-70a5e2668cfb" (UID: "225727ab-44d8-4879-bee7-70a5e2668cfb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:36:15.386859 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.386816 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/225727ab-44d8-4879-bee7-70a5e2668cfb-kube-api-access-b9nzm" (OuterVolumeSpecName: "kube-api-access-b9nzm") pod "225727ab-44d8-4879-bee7-70a5e2668cfb" (UID: "225727ab-44d8-4879-bee7-70a5e2668cfb"). InnerVolumeSpecName "kube-api-access-b9nzm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:36:15.440212 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.440177 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:15.484597 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.484554 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9nzm\" (UniqueName: \"kubernetes.io/projected/225727ab-44d8-4879-bee7-70a5e2668cfb-kube-api-access-b9nzm\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:36:15.484597 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.484589 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/225727ab-44d8-4879-bee7-70a5e2668cfb-plugins-config-volume\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:36:15.519941 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.519896 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:15.705618 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.705582 2560 generic.go:356] "Generic (PLEG): container finished" podID="774cd87d-fc9a-452d-85c5-b758b09eb50d" containerID="7dbf40afd01142bdaa0a34684d7faa445be59e751a8236e064162fb0d00e9551" exitCode=0 Sep 10 13:36:15.705618 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.705623 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-4907abfd-5f6fbc6cbb-6tq2q" Sep 10 13:36:15.705874 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.705667 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-4907abfd-5f6fbc6cbb-6tq2q" event={"ID":"774cd87d-fc9a-452d-85c5-b758b09eb50d","Type":"ContainerDied","Data":"7dbf40afd01142bdaa0a34684d7faa445be59e751a8236e064162fb0d00e9551"} Sep 10 13:36:15.705874 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.705727 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-4907abfd-5f6fbc6cbb-6tq2q" event={"ID":"774cd87d-fc9a-452d-85c5-b758b09eb50d","Type":"ContainerDied","Data":"c0fec5c7e07bcf1a15ea809cc9fd4ee4d53a6189cbf24cef70febafa8566cc48"} Sep 10 13:36:15.705874 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.705762 2560 scope.go:117] "RemoveContainer" containerID="7dbf40afd01142bdaa0a34684d7faa445be59e751a8236e064162fb0d00e9551" Sep 10 13:36:15.707123 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.707096 2560 generic.go:356] "Generic (PLEG): container finished" podID="225727ab-44d8-4879-bee7-70a5e2668cfb" containerID="e45feffd4ff81c0a6e0a8a4c3328339cb90c8daccd07f0d4bfe72f7696864dd4" exitCode=0 Sep 10 13:36:15.707217 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.707153 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-deba50-679c68dccf-g2wgd" Sep 10 13:36:15.707217 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.707202 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-deba50-679c68dccf-g2wgd" event={"ID":"225727ab-44d8-4879-bee7-70a5e2668cfb","Type":"ContainerDied","Data":"e45feffd4ff81c0a6e0a8a4c3328339cb90c8daccd07f0d4bfe72f7696864dd4"} Sep 10 13:36:15.707323 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.707239 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-deba50-679c68dccf-g2wgd" event={"ID":"225727ab-44d8-4879-bee7-70a5e2668cfb","Type":"ContainerDied","Data":"9ef6633fa1fb4a530aa56b3a0004e2235ee100b4a8974ca3222b30815eebf148"} Sep 10 13:36:15.708062 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.708037 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:15.708756 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.708722 2560 generic.go:356] "Generic (PLEG): container finished" podID="a3cd6f5c-2f78-4a22-8a4b-a6911ac25412" containerID="b78aa4bb09c1afd7a4863b255bb9c8e84f218a55dccfbdfdbbf3011cc471ad1d" exitCode=0 Sep 10 13:36:15.708875 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.708759 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-4907abfd-55cf7d755d-cwx4p" Sep 10 13:36:15.708875 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.708779 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-4907abfd-55cf7d755d-cwx4p" event={"ID":"a3cd6f5c-2f78-4a22-8a4b-a6911ac25412","Type":"ContainerDied","Data":"b78aa4bb09c1afd7a4863b255bb9c8e84f218a55dccfbdfdbbf3011cc471ad1d"} Sep 10 13:36:15.708875 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.708817 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-4907abfd-55cf7d755d-cwx4p" event={"ID":"a3cd6f5c-2f78-4a22-8a4b-a6911ac25412","Type":"ContainerDied","Data":"ab310dbf80946a38622a3654d51de0678d02c3a8c0128c5a3bb77ab3df7d8fb4"} Sep 10 13:36:15.716149 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.716125 2560 scope.go:117] "RemoveContainer" containerID="7dbf40afd01142bdaa0a34684d7faa445be59e751a8236e064162fb0d00e9551" Sep 10 13:36:15.716457 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:36:15.716438 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7dbf40afd01142bdaa0a34684d7faa445be59e751a8236e064162fb0d00e9551\": container with ID starting with 7dbf40afd01142bdaa0a34684d7faa445be59e751a8236e064162fb0d00e9551 not found: ID does not exist" containerID="7dbf40afd01142bdaa0a34684d7faa445be59e751a8236e064162fb0d00e9551" Sep 10 13:36:15.716540 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.716463 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dbf40afd01142bdaa0a34684d7faa445be59e751a8236e064162fb0d00e9551"} err="failed to get container status \"7dbf40afd01142bdaa0a34684d7faa445be59e751a8236e064162fb0d00e9551\": rpc error: code = NotFound desc = could not find container \"7dbf40afd01142bdaa0a34684d7faa445be59e751a8236e064162fb0d00e9551\": container with ID starting with 7dbf40afd01142bdaa0a34684d7faa445be59e751a8236e064162fb0d00e9551 not found: ID does not exist" Sep 10 13:36:15.716540 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.716479 2560 scope.go:117] "RemoveContainer" containerID="e45feffd4ff81c0a6e0a8a4c3328339cb90c8daccd07f0d4bfe72f7696864dd4" Sep 10 13:36:15.725946 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.725929 2560 scope.go:117] "RemoveContainer" containerID="e45feffd4ff81c0a6e0a8a4c3328339cb90c8daccd07f0d4bfe72f7696864dd4" Sep 10 13:36:15.726219 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:36:15.726201 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e45feffd4ff81c0a6e0a8a4c3328339cb90c8daccd07f0d4bfe72f7696864dd4\": container with ID starting with e45feffd4ff81c0a6e0a8a4c3328339cb90c8daccd07f0d4bfe72f7696864dd4 not found: ID does not exist" containerID="e45feffd4ff81c0a6e0a8a4c3328339cb90c8daccd07f0d4bfe72f7696864dd4" Sep 10 13:36:15.726296 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.726229 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e45feffd4ff81c0a6e0a8a4c3328339cb90c8daccd07f0d4bfe72f7696864dd4"} err="failed to get container status \"e45feffd4ff81c0a6e0a8a4c3328339cb90c8daccd07f0d4bfe72f7696864dd4\": rpc error: code = NotFound desc = could not find container \"e45feffd4ff81c0a6e0a8a4c3328339cb90c8daccd07f0d4bfe72f7696864dd4\": container with ID starting with e45feffd4ff81c0a6e0a8a4c3328339cb90c8daccd07f0d4bfe72f7696864dd4 not found: ID does not exist" Sep 10 13:36:15.726296 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.726250 2560 scope.go:117] "RemoveContainer" containerID="b78aa4bb09c1afd7a4863b255bb9c8e84f218a55dccfbdfdbbf3011cc471ad1d" Sep 10 13:36:15.734243 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.734228 2560 scope.go:117] "RemoveContainer" containerID="b78aa4bb09c1afd7a4863b255bb9c8e84f218a55dccfbdfdbbf3011cc471ad1d" Sep 10 13:36:15.734560 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:36:15.734529 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b78aa4bb09c1afd7a4863b255bb9c8e84f218a55dccfbdfdbbf3011cc471ad1d\": container with ID starting with b78aa4bb09c1afd7a4863b255bb9c8e84f218a55dccfbdfdbbf3011cc471ad1d not found: ID does not exist" containerID="b78aa4bb09c1afd7a4863b255bb9c8e84f218a55dccfbdfdbbf3011cc471ad1d" Sep 10 13:36:15.734560 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.734552 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b78aa4bb09c1afd7a4863b255bb9c8e84f218a55dccfbdfdbbf3011cc471ad1d"} err="failed to get container status \"b78aa4bb09c1afd7a4863b255bb9c8e84f218a55dccfbdfdbbf3011cc471ad1d\": rpc error: code = NotFound desc = could not find container \"b78aa4bb09c1afd7a4863b255bb9c8e84f218a55dccfbdfdbbf3011cc471ad1d\": container with ID starting with b78aa4bb09c1afd7a4863b255bb9c8e84f218a55dccfbdfdbbf3011cc471ad1d not found: ID does not exist" Sep 10 13:36:15.740915 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.740895 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-4907abfd-5f6fbc6cbb-6tq2q"] Sep 10 13:36:15.749964 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.749941 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-4907abfd-5f6fbc6cbb-6tq2q"] Sep 10 13:36:15.780708 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.780672 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-4907abfd-55cf7d755d-cwx4p"] Sep 10 13:36:15.783133 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.783103 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-4907abfd-55cf7d755d-cwx4p"] Sep 10 13:36:15.814292 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.814261 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-deba50-679c68dccf-g2wgd"] Sep 10 13:36:15.821651 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.821622 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-deba50-679c68dccf-g2wgd"] Sep 10 13:36:15.833855 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:15.833830 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:16.107921 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.107837 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2"] Sep 10 13:36:16.108252 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.108240 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3cd6f5c-2f78-4a22-8a4b-a6911ac25412" containerName="payload-pre-processing" Sep 10 13:36:16.108252 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.108253 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3cd6f5c-2f78-4a22-8a4b-a6911ac25412" containerName="payload-pre-processing" Sep 10 13:36:16.108341 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.108272 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="774cd87d-fc9a-452d-85c5-b758b09eb50d" containerName="payload-processing" Sep 10 13:36:16.108341 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.108277 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="774cd87d-fc9a-452d-85c5-b758b09eb50d" containerName="payload-processing" Sep 10 13:36:16.108341 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.108290 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="225727ab-44d8-4879-bee7-70a5e2668cfb" containerName="payload-processing" Sep 10 13:36:16.108341 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.108295 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="225727ab-44d8-4879-bee7-70a5e2668cfb" containerName="payload-processing" Sep 10 13:36:16.108514 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.108357 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="225727ab-44d8-4879-bee7-70a5e2668cfb" containerName="payload-processing" Sep 10 13:36:16.108514 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.108365 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3cd6f5c-2f78-4a22-8a4b-a6911ac25412" containerName="payload-pre-processing" Sep 10 13:36:16.108514 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.108376 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="774cd87d-fc9a-452d-85c5-b758b09eb50d" containerName="payload-processing" Sep 10 13:36:16.113026 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.113006 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:16.119712 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.119685 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-95ae6288-openshift-default-dockercfg-qrl6p\"" Sep 10 13:36:16.172135 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.172099 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:16.194695 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.191952 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-istio-envoy\") pod \"e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2\" (UID: \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\") " pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:16.194695 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.192078 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-istiod-ca-cert\") pod \"e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2\" (UID: \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\") " pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:16.194695 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.192112 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-istio-token\") pod \"e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2\" (UID: \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\") " pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:16.194695 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.192138 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-istio-podinfo\") pod \"e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2\" (UID: \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\") " pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:16.194695 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.192185 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-workload-certs\") pod \"e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2\" (UID: \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\") " pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:16.194695 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.192227 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nksll\" (UniqueName: \"kubernetes.io/projected/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-kube-api-access-nksll\") pod \"e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2\" (UID: \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\") " pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:16.194695 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.192277 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-credential-socket\") pod \"e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2\" (UID: \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\") " pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:16.194695 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.192319 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-istio-data\") pod \"e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2\" (UID: \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\") " pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:16.194695 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.192342 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-workload-socket\") pod \"e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2\" (UID: \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\") " pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:16.216676 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.216634 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2"] Sep 10 13:36:16.293608 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.293568 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-istio-data\") pod \"e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2\" (UID: \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\") " pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:16.293608 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.293611 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-workload-socket\") pod \"e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2\" (UID: \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\") " pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:16.293892 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.293654 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-istio-envoy\") pod \"e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2\" (UID: \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\") " pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:16.293892 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.293706 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-istiod-ca-cert\") pod \"e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2\" (UID: \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\") " pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:16.293892 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.293817 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-istio-token\") pod \"e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2\" (UID: \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\") " pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:16.293892 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.293865 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-istio-podinfo\") pod \"e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2\" (UID: \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\") " pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:16.294117 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.293924 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-workload-certs\") pod \"e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2\" (UID: \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\") " pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:16.294117 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.293972 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nksll\" (UniqueName: \"kubernetes.io/projected/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-kube-api-access-nksll\") pod \"e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2\" (UID: \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\") " pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:16.294117 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.294014 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-istio-data\") pod \"e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2\" (UID: \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\") " pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:16.294117 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.294026 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-credential-socket\") pod \"e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2\" (UID: \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\") " pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:16.294117 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.294038 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-workload-socket\") pod \"e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2\" (UID: \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\") " pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:16.294386 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.294286 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-credential-socket\") pod \"e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2\" (UID: \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\") " pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:16.294386 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.294374 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-workload-certs\") pod \"e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2\" (UID: \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\") " pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:16.294532 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.294464 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-istiod-ca-cert\") pod \"e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2\" (UID: \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\") " pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:16.296031 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.296013 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-istio-envoy\") pod \"e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2\" (UID: \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\") " pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:16.296285 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.296265 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-istio-podinfo\") pod \"e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2\" (UID: \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\") " pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:16.306150 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.306121 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-istio-token\") pod \"e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2\" (UID: \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\") " pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:16.306290 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.306208 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nksll\" (UniqueName: \"kubernetes.io/projected/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-kube-api-access-nksll\") pod \"e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2\" (UID: \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\") " pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:16.382607 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.382524 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:16.426176 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.426141 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:16.564674 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.564641 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="225727ab-44d8-4879-bee7-70a5e2668cfb" path="/var/lib/kubelet/pods/225727ab-44d8-4879-bee7-70a5e2668cfb/volumes" Sep 10 13:36:16.565020 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.565007 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="774cd87d-fc9a-452d-85c5-b758b09eb50d" path="/var/lib/kubelet/pods/774cd87d-fc9a-452d-85c5-b758b09eb50d/volumes" Sep 10 13:36:16.565323 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.565312 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3cd6f5c-2f78-4a22-8a4b-a6911ac25412" path="/var/lib/kubelet/pods/a3cd6f5c-2f78-4a22-8a4b-a6911ac25412/volumes" Sep 10 13:36:16.714222 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.714195 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2"] Sep 10 13:36:16.716053 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:36:16.716022 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89ececc1_da3b_4ff4_a6cb_8cf608f0210b.slice/crio-610ad242948498a6fc60f1a6ee3556a6553371bf3156a745a736622eacab64d3 WatchSource:0}: Error finding container 610ad242948498a6fc60f1a6ee3556a6553371bf3156a745a736622eacab64d3: Status 404 returned error can't find the container with id 610ad242948498a6fc60f1a6ee3556a6553371bf3156a745a736622eacab64d3 Sep 10 13:36:16.718370 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.718337 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:36:16.718486 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.718426 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:36:16.718486 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:16.718469 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:36:17.727442 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:17.727388 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" event={"ID":"89ececc1-da3b-4ff4-a6cb-8cf608f0210b","Type":"ContainerStarted","Data":"d506f261a251ba5cf7a72a39fb4676edb17c592ca9088f58d38e44d5a69be456"} Sep 10 13:36:17.727843 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:17.727460 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" event={"ID":"89ececc1-da3b-4ff4-a6cb-8cf608f0210b","Type":"ContainerStarted","Data":"610ad242948498a6fc60f1a6ee3556a6553371bf3156a745a736622eacab64d3"} Sep 10 13:36:17.827153 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:17.827097 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" podStartSLOduration=1.8270790080000001 podStartE2EDuration="1.827079008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:36:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:36:17.82426644 +0000 UTC m=+1087.718983859" watchObservedRunningTime="2026-09-10 13:36:17.827079008 +0000 UTC m=+1087.721796425" Sep 10 13:36:18.426586 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:18.426549 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:18.432424 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:18.432383 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:18.733916 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:18.733824 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:18.734769 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:18.734752 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:19.186007 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:19.185974 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:19.283585 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:19.283550 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:19.689137 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:19.689098 2560 patch_prober.go:28] interesting pod/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Sep 10 13:36:19.689329 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:19.689162 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" podUID="1017b5b4-ae1f-4925-b29b-3a87740fc0a5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 13:36:20.514726 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.514643 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:36:20.537012 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.536980 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-75s2c\" (UniqueName: \"kubernetes.io/projected/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-kube-api-access-75s2c\") pod \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\" (UID: \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\") " Sep 10 13:36:20.537012 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.537024 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-credential-socket\") pod \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\" (UID: \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\") " Sep 10 13:36:20.537277 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.537065 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-workload-socket\") pod \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\" (UID: \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\") " Sep 10 13:36:20.537277 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.537178 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-istio-podinfo\") pod \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\" (UID: \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\") " Sep 10 13:36:20.537277 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.537243 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-istio-envoy\") pod \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\" (UID: \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\") " Sep 10 13:36:20.537277 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.537273 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-workload-certs\") pod \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\" (UID: \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\") " Sep 10 13:36:20.537513 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.537355 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-istio-data\") pod \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\" (UID: \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\") " Sep 10 13:36:20.537513 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.537416 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-istio-token\") pod \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\" (UID: \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\") " Sep 10 13:36:20.537513 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.537446 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-istiod-ca-cert\") pod \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\" (UID: \"1017b5b4-ae1f-4925-b29b-3a87740fc0a5\") " Sep 10 13:36:20.537513 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.537449 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1017b5b4-ae1f-4925-b29b-3a87740fc0a5" (UID: "1017b5b4-ae1f-4925-b29b-3a87740fc0a5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:36:20.537800 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.537777 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1017b5b4-ae1f-4925-b29b-3a87740fc0a5" (UID: "1017b5b4-ae1f-4925-b29b-3a87740fc0a5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:36:20.537879 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.537833 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-istio-data" (OuterVolumeSpecName: "istio-data") pod "1017b5b4-ae1f-4925-b29b-3a87740fc0a5" (UID: "1017b5b4-ae1f-4925-b29b-3a87740fc0a5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:36:20.537941 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.537883 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1017b5b4-ae1f-4925-b29b-3a87740fc0a5" (UID: "1017b5b4-ae1f-4925-b29b-3a87740fc0a5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:36:20.537941 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.537897 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1017b5b4-ae1f-4925-b29b-3a87740fc0a5" (UID: "1017b5b4-ae1f-4925-b29b-3a87740fc0a5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:36:20.538180 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.538161 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-credential-socket\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:36:20.538275 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.538207 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-workload-socket\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:36:20.538275 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.538225 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-workload-certs\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:36:20.538275 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.538238 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-istio-data\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:36:20.538275 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.538269 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-istiod-ca-cert\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:36:20.539997 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.539965 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1017b5b4-ae1f-4925-b29b-3a87740fc0a5" (UID: "1017b5b4-ae1f-4925-b29b-3a87740fc0a5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:36:20.540108 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.540024 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1017b5b4-ae1f-4925-b29b-3a87740fc0a5" (UID: "1017b5b4-ae1f-4925-b29b-3a87740fc0a5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 13:36:20.540108 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.540061 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-kube-api-access-75s2c" (OuterVolumeSpecName: "kube-api-access-75s2c") pod "1017b5b4-ae1f-4925-b29b-3a87740fc0a5" (UID: "1017b5b4-ae1f-4925-b29b-3a87740fc0a5"). InnerVolumeSpecName "kube-api-access-75s2c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:36:20.540218 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.540157 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-istio-token" (OuterVolumeSpecName: "istio-token") pod "1017b5b4-ae1f-4925-b29b-3a87740fc0a5" (UID: "1017b5b4-ae1f-4925-b29b-3a87740fc0a5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:36:20.547929 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.547682 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:20.639639 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.639514 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75s2c\" (UniqueName: \"kubernetes.io/projected/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-kube-api-access-75s2c\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:36:20.639639 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.639548 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-istio-podinfo\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:36:20.639639 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.639564 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-istio-envoy\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:36:20.639639 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.639579 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1017b5b4-ae1f-4925-b29b-3a87740fc0a5-istio-token\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:36:20.669668 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.669613 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:20.748938 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.748900 2560 generic.go:356] "Generic (PLEG): container finished" podID="1017b5b4-ae1f-4925-b29b-3a87740fc0a5" containerID="2670009207e8450e184a98384a11a6244883f3553bee61ae4b0f32d2c24af036" exitCode=0 Sep 10 13:36:20.748938 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.748924 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" event={"ID":"1017b5b4-ae1f-4925-b29b-3a87740fc0a5","Type":"ContainerDied","Data":"2670009207e8450e184a98384a11a6244883f3553bee61ae4b0f32d2c24af036"} Sep 10 13:36:20.749174 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.748953 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" Sep 10 13:36:20.749174 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.748980 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv" event={"ID":"1017b5b4-ae1f-4925-b29b-3a87740fc0a5","Type":"ContainerDied","Data":"3bb22c8fe588c1ede9700ee86be2801738018c4ef7ea9849c22a0115787a1b3a"} Sep 10 13:36:20.749174 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.749002 2560 scope.go:117] "RemoveContainer" containerID="2670009207e8450e184a98384a11a6244883f3553bee61ae4b0f32d2c24af036" Sep 10 13:36:20.758661 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.758637 2560 scope.go:117] "RemoveContainer" containerID="1d4bdd141cfbe33127bfb8b3c07ea11de257d40e78e1ab73404c7fe8896be614" Sep 10 13:36:20.768069 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.768013 2560 scope.go:117] "RemoveContainer" containerID="2670009207e8450e184a98384a11a6244883f3553bee61ae4b0f32d2c24af036" Sep 10 13:36:20.768286 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:36:20.768262 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2670009207e8450e184a98384a11a6244883f3553bee61ae4b0f32d2c24af036\": container with ID starting with 2670009207e8450e184a98384a11a6244883f3553bee61ae4b0f32d2c24af036 not found: ID does not exist" containerID="2670009207e8450e184a98384a11a6244883f3553bee61ae4b0f32d2c24af036" Sep 10 13:36:20.768366 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.768301 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2670009207e8450e184a98384a11a6244883f3553bee61ae4b0f32d2c24af036"} err="failed to get container status \"2670009207e8450e184a98384a11a6244883f3553bee61ae4b0f32d2c24af036\": rpc error: code = NotFound desc = could not find container \"2670009207e8450e184a98384a11a6244883f3553bee61ae4b0f32d2c24af036\": container with ID starting with 2670009207e8450e184a98384a11a6244883f3553bee61ae4b0f32d2c24af036 not found: ID does not exist" Sep 10 13:36:20.768366 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.768328 2560 scope.go:117] "RemoveContainer" containerID="1d4bdd141cfbe33127bfb8b3c07ea11de257d40e78e1ab73404c7fe8896be614" Sep 10 13:36:20.768627 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:36:20.768603 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d4bdd141cfbe33127bfb8b3c07ea11de257d40e78e1ab73404c7fe8896be614\": container with ID starting with 1d4bdd141cfbe33127bfb8b3c07ea11de257d40e78e1ab73404c7fe8896be614 not found: ID does not exist" containerID="1d4bdd141cfbe33127bfb8b3c07ea11de257d40e78e1ab73404c7fe8896be614" Sep 10 13:36:20.768686 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.768638 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d4bdd141cfbe33127bfb8b3c07ea11de257d40e78e1ab73404c7fe8896be614"} err="failed to get container status \"1d4bdd141cfbe33127bfb8b3c07ea11de257d40e78e1ab73404c7fe8896be614\": rpc error: code = NotFound desc = could not find container \"1d4bdd141cfbe33127bfb8b3c07ea11de257d40e78e1ab73404c7fe8896be614\": container with ID starting with 1d4bdd141cfbe33127bfb8b3c07ea11de257d40e78e1ab73404c7fe8896be614 not found: ID does not exist" Sep 10 13:36:20.775962 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.775930 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv"] Sep 10 13:36:20.778985 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:20.778960 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-4907abfd-gw-openshift-default-8485476444-swmbv"] Sep 10 13:36:21.198944 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:21.198912 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:21.583603 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:21.583510 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:21.689134 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:21.689100 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:21.805390 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:21.805351 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-95ae6288-67867d978b-v5854"] Sep 10 13:36:21.805856 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:21.805841 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1017b5b4-ae1f-4925-b29b-3a87740fc0a5" containerName="istio-proxy" Sep 10 13:36:21.805901 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:21.805860 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1017b5b4-ae1f-4925-b29b-3a87740fc0a5" containerName="istio-proxy" Sep 10 13:36:21.805901 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:21.805876 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1017b5b4-ae1f-4925-b29b-3a87740fc0a5" containerName="istio-proxy" Sep 10 13:36:21.805901 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:21.805882 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1017b5b4-ae1f-4925-b29b-3a87740fc0a5" containerName="istio-proxy" Sep 10 13:36:21.805995 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:21.805952 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="1017b5b4-ae1f-4925-b29b-3a87740fc0a5" containerName="istio-proxy" Sep 10 13:36:21.805995 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:21.805961 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="1017b5b4-ae1f-4925-b29b-3a87740fc0a5" containerName="istio-proxy" Sep 10 13:36:21.810648 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:21.810630 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-95ae6288-67867d978b-v5854" Sep 10 13:36:21.816773 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:21.816753 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-95ae6288\"" Sep 10 13:36:21.817212 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:21.816979 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-95ae6288-dockercfg-sb9gc\"" Sep 10 13:36:21.842240 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:21.842154 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-95ae6288-67867d978b-v5854"] Sep 10 13:36:21.851651 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:21.851617 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/773e5bdc-f53c-4109-bbc9-447290933152-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-95ae6288-67867d978b-v5854\" (UID: \"773e5bdc-f53c-4109-bbc9-447290933152\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-95ae6288-67867d978b-v5854" Sep 10 13:36:21.851822 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:21.851725 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgh6t\" (UniqueName: \"kubernetes.io/projected/773e5bdc-f53c-4109-bbc9-447290933152-kube-api-access-qgh6t\") pod \"payload-pre-processing-e2e-ait-95ae6288-67867d978b-v5854\" (UID: \"773e5bdc-f53c-4109-bbc9-447290933152\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-95ae6288-67867d978b-v5854" Sep 10 13:36:21.887271 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:21.887228 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-95ae6288-79dd4886c-9cmns"] Sep 10 13:36:21.891787 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:21.891756 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-95ae6288-79dd4886c-9cmns" Sep 10 13:36:21.905535 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:21.905501 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-95ae6288-79dd4886c-9cmns"] Sep 10 13:36:21.953086 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:21.953049 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/773e5bdc-f53c-4109-bbc9-447290933152-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-95ae6288-67867d978b-v5854\" (UID: \"773e5bdc-f53c-4109-bbc9-447290933152\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-95ae6288-67867d978b-v5854" Sep 10 13:36:21.953307 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:21.953139 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54972101-69fe-4982-945d-4db862573444-plugins-config-volume\") pod \"payload-processing-e2e-ait-95ae6288-79dd4886c-9cmns\" (UID: \"54972101-69fe-4982-945d-4db862573444\") " pod="openshift-ingress/payload-processing-e2e-ait-95ae6288-79dd4886c-9cmns" Sep 10 13:36:21.953307 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:21.953168 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgh6t\" (UniqueName: \"kubernetes.io/projected/773e5bdc-f53c-4109-bbc9-447290933152-kube-api-access-qgh6t\") pod \"payload-pre-processing-e2e-ait-95ae6288-67867d978b-v5854\" (UID: \"773e5bdc-f53c-4109-bbc9-447290933152\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-95ae6288-67867d978b-v5854" Sep 10 13:36:21.953307 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:21.953281 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wbkx\" (UniqueName: \"kubernetes.io/projected/54972101-69fe-4982-945d-4db862573444-kube-api-access-2wbkx\") pod \"payload-processing-e2e-ait-95ae6288-79dd4886c-9cmns\" (UID: \"54972101-69fe-4982-945d-4db862573444\") " pod="openshift-ingress/payload-processing-e2e-ait-95ae6288-79dd4886c-9cmns" Sep 10 13:36:21.953844 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:21.953826 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/773e5bdc-f53c-4109-bbc9-447290933152-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-95ae6288-67867d978b-v5854\" (UID: \"773e5bdc-f53c-4109-bbc9-447290933152\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-95ae6288-67867d978b-v5854" Sep 10 13:36:21.962889 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:21.962855 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgh6t\" (UniqueName: \"kubernetes.io/projected/773e5bdc-f53c-4109-bbc9-447290933152-kube-api-access-qgh6t\") pod \"payload-pre-processing-e2e-ait-95ae6288-67867d978b-v5854\" (UID: \"773e5bdc-f53c-4109-bbc9-447290933152\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-95ae6288-67867d978b-v5854" Sep 10 13:36:22.054003 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:22.053968 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wbkx\" (UniqueName: \"kubernetes.io/projected/54972101-69fe-4982-945d-4db862573444-kube-api-access-2wbkx\") pod \"payload-processing-e2e-ait-95ae6288-79dd4886c-9cmns\" (UID: \"54972101-69fe-4982-945d-4db862573444\") " pod="openshift-ingress/payload-processing-e2e-ait-95ae6288-79dd4886c-9cmns" Sep 10 13:36:22.054190 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:22.054071 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54972101-69fe-4982-945d-4db862573444-plugins-config-volume\") pod \"payload-processing-e2e-ait-95ae6288-79dd4886c-9cmns\" (UID: \"54972101-69fe-4982-945d-4db862573444\") " pod="openshift-ingress/payload-processing-e2e-ait-95ae6288-79dd4886c-9cmns" Sep 10 13:36:22.054794 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:22.054774 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54972101-69fe-4982-945d-4db862573444-plugins-config-volume\") pod \"payload-processing-e2e-ait-95ae6288-79dd4886c-9cmns\" (UID: \"54972101-69fe-4982-945d-4db862573444\") " pod="openshift-ingress/payload-processing-e2e-ait-95ae6288-79dd4886c-9cmns" Sep 10 13:36:22.066007 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:22.065979 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wbkx\" (UniqueName: \"kubernetes.io/projected/54972101-69fe-4982-945d-4db862573444-kube-api-access-2wbkx\") pod \"payload-processing-e2e-ait-95ae6288-79dd4886c-9cmns\" (UID: \"54972101-69fe-4982-945d-4db862573444\") " pod="openshift-ingress/payload-processing-e2e-ait-95ae6288-79dd4886c-9cmns" Sep 10 13:36:22.121435 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:22.121315 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-95ae6288-67867d978b-v5854" Sep 10 13:36:22.205780 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:22.205748 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-95ae6288-79dd4886c-9cmns" Sep 10 13:36:22.402887 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:22.402862 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-95ae6288-67867d978b-v5854"] Sep 10 13:36:22.404577 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:36:22.404553 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod773e5bdc_f53c_4109_bbc9_447290933152.slice/crio-f4176cfd4c6b48730d45fd1adf4fcb4c93e17c9a7269969e02d95f0cbae587b3 WatchSource:0}: Error finding container f4176cfd4c6b48730d45fd1adf4fcb4c93e17c9a7269969e02d95f0cbae587b3: Status 404 returned error can't find the container with id f4176cfd4c6b48730d45fd1adf4fcb4c93e17c9a7269969e02d95f0cbae587b3 Sep 10 13:36:22.488845 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:22.488818 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-95ae6288-79dd4886c-9cmns"] Sep 10 13:36:22.491195 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:36:22.491163 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54972101_69fe_4982_945d_4db862573444.slice/crio-3b0da71db48dbc4b3f0bb667b035b129387866093dc9948baac295a9ebd1b8e8 WatchSource:0}: Error finding container 3b0da71db48dbc4b3f0bb667b035b129387866093dc9948baac295a9ebd1b8e8: Status 404 returned error can't find the container with id 3b0da71db48dbc4b3f0bb667b035b129387866093dc9948baac295a9ebd1b8e8 Sep 10 13:36:22.565490 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:22.565432 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1017b5b4-ae1f-4925-b29b-3a87740fc0a5" path="/var/lib/kubelet/pods/1017b5b4-ae1f-4925-b29b-3a87740fc0a5/volumes" Sep 10 13:36:22.765237 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:22.765197 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-95ae6288-79dd4886c-9cmns" event={"ID":"54972101-69fe-4982-945d-4db862573444","Type":"ContainerStarted","Data":"538ee72895ef22a5f4b5c9973ea42c2b5298f7b411fa4364a6facec5fc1bf9ce"} Sep 10 13:36:22.765731 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:22.765268 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-95ae6288-79dd4886c-9cmns" Sep 10 13:36:22.765731 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:22.765295 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-95ae6288-79dd4886c-9cmns" event={"ID":"54972101-69fe-4982-945d-4db862573444","Type":"ContainerStarted","Data":"3b0da71db48dbc4b3f0bb667b035b129387866093dc9948baac295a9ebd1b8e8"} Sep 10 13:36:22.767034 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:22.767012 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-95ae6288-67867d978b-v5854" event={"ID":"773e5bdc-f53c-4109-bbc9-447290933152","Type":"ContainerStarted","Data":"b6a40951850f269ae7045151c380420437ede3d81e2dc2f6db1878755b99f73b"} Sep 10 13:36:22.767163 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:22.767053 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-95ae6288-67867d978b-v5854" event={"ID":"773e5bdc-f53c-4109-bbc9-447290933152","Type":"ContainerStarted","Data":"f4176cfd4c6b48730d45fd1adf4fcb4c93e17c9a7269969e02d95f0cbae587b3"} Sep 10 13:36:22.767163 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:22.767151 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-95ae6288-67867d978b-v5854" Sep 10 13:36:22.793034 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:22.792976 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-95ae6288-79dd4886c-9cmns" podStartSLOduration=1.792960918 podStartE2EDuration="1.792960918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:36:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:36:22.792113805 +0000 UTC m=+1092.686831236" watchObservedRunningTime="2026-09-10 13:36:22.792960918 +0000 UTC m=+1092.687678335" Sep 10 13:36:22.832209 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:22.832144 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-95ae6288-67867d978b-v5854" podStartSLOduration=1.8321263829999999 podStartE2EDuration="1.832126383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:36:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:36:22.829518024 +0000 UTC m=+1092.724235443" watchObservedRunningTime="2026-09-10 13:36:22.832126383 +0000 UTC m=+1092.726843832" Sep 10 13:36:24.587147 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:24.587114 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:25.003126 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:25.003095 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:25.184026 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:25.183993 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:25.545628 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:25.545584 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:25.866485 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:25.866372 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:25.968212 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:25.968176 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:27.998591 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:27.998553 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:28.100231 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:28.100180 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:29.113701 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:29.113670 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:29.753096 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:29.753056 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:29.867270 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:29.867238 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:30.662948 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:30.662907 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:30.762809 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:30.762773 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:30.942064 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:30.942028 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:31.296949 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:31.296871 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:31.396503 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:31.396463 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:31.441745 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:31.441688 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-3c32b1-86f4465b45-s8jd6"] Sep 10 13:36:31.446838 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:31.446805 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-3c32b1-86f4465b45-s8jd6" Sep 10 13:36:31.450529 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:31.450505 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-3c32b1\"" Sep 10 13:36:31.450973 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:31.450778 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-3c32b1-dockercfg-p59zr\"" Sep 10 13:36:31.470855 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:31.468936 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-3c32b1-86f4465b45-s8jd6"] Sep 10 13:36:31.562124 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:31.562029 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pr7hb\" (UniqueName: \"kubernetes.io/projected/f193a6dd-401a-4763-bc0c-3ee20a960046-kube-api-access-pr7hb\") pod \"payload-processing-e2e-explicit-3c32b1-86f4465b45-s8jd6\" (UID: \"f193a6dd-401a-4763-bc0c-3ee20a960046\") " pod="openshift-ingress/payload-processing-e2e-explicit-3c32b1-86f4465b45-s8jd6" Sep 10 13:36:31.562507 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:31.562481 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f193a6dd-401a-4763-bc0c-3ee20a960046-plugins-config-volume\") pod \"payload-processing-e2e-explicit-3c32b1-86f4465b45-s8jd6\" (UID: \"f193a6dd-401a-4763-bc0c-3ee20a960046\") " pod="openshift-ingress/payload-processing-e2e-explicit-3c32b1-86f4465b45-s8jd6" Sep 10 13:36:31.663843 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:31.663786 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f193a6dd-401a-4763-bc0c-3ee20a960046-plugins-config-volume\") pod \"payload-processing-e2e-explicit-3c32b1-86f4465b45-s8jd6\" (UID: \"f193a6dd-401a-4763-bc0c-3ee20a960046\") " pod="openshift-ingress/payload-processing-e2e-explicit-3c32b1-86f4465b45-s8jd6" Sep 10 13:36:31.664326 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:31.663931 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pr7hb\" (UniqueName: \"kubernetes.io/projected/f193a6dd-401a-4763-bc0c-3ee20a960046-kube-api-access-pr7hb\") pod \"payload-processing-e2e-explicit-3c32b1-86f4465b45-s8jd6\" (UID: \"f193a6dd-401a-4763-bc0c-3ee20a960046\") " pod="openshift-ingress/payload-processing-e2e-explicit-3c32b1-86f4465b45-s8jd6" Sep 10 13:36:31.665305 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:31.665270 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f193a6dd-401a-4763-bc0c-3ee20a960046-plugins-config-volume\") pod \"payload-processing-e2e-explicit-3c32b1-86f4465b45-s8jd6\" (UID: \"f193a6dd-401a-4763-bc0c-3ee20a960046\") " pod="openshift-ingress/payload-processing-e2e-explicit-3c32b1-86f4465b45-s8jd6" Sep 10 13:36:31.675578 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:31.675550 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pr7hb\" (UniqueName: \"kubernetes.io/projected/f193a6dd-401a-4763-bc0c-3ee20a960046-kube-api-access-pr7hb\") pod \"payload-processing-e2e-explicit-3c32b1-86f4465b45-s8jd6\" (UID: \"f193a6dd-401a-4763-bc0c-3ee20a960046\") " pod="openshift-ingress/payload-processing-e2e-explicit-3c32b1-86f4465b45-s8jd6" Sep 10 13:36:31.772338 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:31.772294 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-3c32b1-86f4465b45-s8jd6" Sep 10 13:36:32.055575 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:32.055263 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-3c32b1-86f4465b45-s8jd6"] Sep 10 13:36:32.832661 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:32.832616 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-3c32b1-86f4465b45-s8jd6" event={"ID":"f193a6dd-401a-4763-bc0c-3ee20a960046","Type":"ContainerStarted","Data":"5a4cbae318d9a5e9cb6dd68f05628239a493aba4244e045c642d14e7ea5cda40"} Sep 10 13:36:32.833105 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:32.832684 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-3c32b1-86f4465b45-s8jd6" event={"ID":"f193a6dd-401a-4763-bc0c-3ee20a960046","Type":"ContainerStarted","Data":"34153574e849d03f9c5e619633b009a9bdb1a7555a250c0a2528ec59a18afce4"} Sep 10 13:36:32.833105 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:32.832728 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-3c32b1-86f4465b45-s8jd6" Sep 10 13:36:32.857387 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:32.857309 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-3c32b1-86f4465b45-s8jd6" podStartSLOduration=1.857288305 podStartE2EDuration="1.857288305s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:36:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:36:32.854441674 +0000 UTC m=+1102.749159119" watchObservedRunningTime="2026-09-10 13:36:32.857288305 +0000 UTC m=+1102.752005726" Sep 10 13:36:33.775081 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:33.775045 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-95ae6288-79dd4886c-9cmns" Sep 10 13:36:33.775268 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:33.775129 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-95ae6288-67867d978b-v5854" Sep 10 13:36:38.873902 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:38.873798 2560 generic.go:356] "Generic (PLEG): container finished" podID="89ececc1-da3b-4ff4-a6cb-8cf608f0210b" containerID="d506f261a251ba5cf7a72a39fb4676edb17c592ca9088f58d38e44d5a69be456" exitCode=0 Sep 10 13:36:38.873902 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:38.873869 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" event={"ID":"89ececc1-da3b-4ff4-a6cb-8cf608f0210b","Type":"ContainerDied","Data":"d506f261a251ba5cf7a72a39fb4676edb17c592ca9088f58d38e44d5a69be456"} Sep 10 13:36:38.874344 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:38.874282 2560 scope.go:117] "RemoveContainer" containerID="d506f261a251ba5cf7a72a39fb4676edb17c592ca9088f58d38e44d5a69be456" Sep 10 13:36:38.875363 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:38.875339 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:36:38.875438 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:38.875426 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:36:38.875479 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:38.875462 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:36:39.883863 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:39.883826 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" event={"ID":"89ececc1-da3b-4ff4-a6cb-8cf608f0210b","Type":"ContainerStarted","Data":"ee693ea0895a2f3acf6879a158b752370a397ceb8af1fd77423cd5dc2a702c8a"} Sep 10 13:36:40.426453 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:40.426391 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:40.426662 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:40.426638 2560 patch_prober.go:28] interesting pod/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Sep 10 13:36:40.426716 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:40.426697 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" podUID="89ececc1-da3b-4ff4-a6cb-8cf608f0210b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Sep 10 13:36:41.426904 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:41.426862 2560 patch_prober.go:28] interesting pod/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Sep 10 13:36:41.427323 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:41.426925 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" podUID="89ececc1-da3b-4ff4-a6cb-8cf608f0210b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Sep 10 13:36:42.152689 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:42.152648 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:42.427284 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:42.427247 2560 patch_prober.go:28] interesting pod/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Sep 10 13:36:42.427706 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:42.427317 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" podUID="89ececc1-da3b-4ff4-a6cb-8cf608f0210b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Sep 10 13:36:43.426879 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:43.426837 2560 patch_prober.go:28] interesting pod/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Sep 10 13:36:43.427078 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:43.426915 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" podUID="89ececc1-da3b-4ff4-a6cb-8cf608f0210b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Sep 10 13:36:43.842645 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:43.842555 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-3c32b1-86f4465b45-s8jd6" Sep 10 13:36:44.427003 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:44.426955 2560 patch_prober.go:28] interesting pod/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Sep 10 13:36:44.427218 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:44.427036 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" podUID="89ececc1-da3b-4ff4-a6cb-8cf608f0210b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Sep 10 13:36:45.430666 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:45.430633 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:45.431075 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:45.430913 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:45.431587 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:45.431571 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:51.248783 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.248753 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:51.277335 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.277301 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-95ae6288-79dd4886c-9cmns"] Sep 10 13:36:51.277559 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.277532 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-95ae6288-79dd4886c-9cmns" podUID="54972101-69fe-4982-945d-4db862573444" containerName="payload-processing" containerID="cri-o://538ee72895ef22a5f4b5c9973ea42c2b5298f7b411fa4364a6facec5fc1bf9ce" gracePeriod=30 Sep 10 13:36:51.290461 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.290428 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-95ae6288-67867d978b-v5854"] Sep 10 13:36:51.290721 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.290694 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-95ae6288-67867d978b-v5854" podUID="773e5bdc-f53c-4109-bbc9-447290933152" containerName="payload-pre-processing" containerID="cri-o://b6a40951850f269ae7045151c380420437ede3d81e2dc2f6db1878755b99f73b" gracePeriod=30 Sep 10 13:36:51.386188 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.386143 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:51.530517 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.530481 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:51.581442 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.581390 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-95ae6288-79dd4886c-9cmns" Sep 10 13:36:51.604671 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.604647 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-95ae6288-67867d978b-v5854" Sep 10 13:36:51.655092 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.655061 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:51.656061 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.656036 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/773e5bdc-f53c-4109-bbc9-447290933152-plugins-config-volume\") pod \"773e5bdc-f53c-4109-bbc9-447290933152\" (UID: \"773e5bdc-f53c-4109-bbc9-447290933152\") " Sep 10 13:36:51.656194 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.656068 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54972101-69fe-4982-945d-4db862573444-plugins-config-volume\") pod \"54972101-69fe-4982-945d-4db862573444\" (UID: \"54972101-69fe-4982-945d-4db862573444\") " Sep 10 13:36:51.656194 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.656139 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2wbkx\" (UniqueName: \"kubernetes.io/projected/54972101-69fe-4982-945d-4db862573444-kube-api-access-2wbkx\") pod \"54972101-69fe-4982-945d-4db862573444\" (UID: \"54972101-69fe-4982-945d-4db862573444\") " Sep 10 13:36:51.656194 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.656169 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qgh6t\" (UniqueName: \"kubernetes.io/projected/773e5bdc-f53c-4109-bbc9-447290933152-kube-api-access-qgh6t\") pod \"773e5bdc-f53c-4109-bbc9-447290933152\" (UID: \"773e5bdc-f53c-4109-bbc9-447290933152\") " Sep 10 13:36:51.656480 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.656439 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/773e5bdc-f53c-4109-bbc9-447290933152-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "773e5bdc-f53c-4109-bbc9-447290933152" (UID: "773e5bdc-f53c-4109-bbc9-447290933152"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:36:51.656573 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.656547 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54972101-69fe-4982-945d-4db862573444-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "54972101-69fe-4982-945d-4db862573444" (UID: "54972101-69fe-4982-945d-4db862573444"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:36:51.656641 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.656571 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/773e5bdc-f53c-4109-bbc9-447290933152-plugins-config-volume\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:36:51.658362 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.658340 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54972101-69fe-4982-945d-4db862573444-kube-api-access-2wbkx" (OuterVolumeSpecName: "kube-api-access-2wbkx") pod "54972101-69fe-4982-945d-4db862573444" (UID: "54972101-69fe-4982-945d-4db862573444"). InnerVolumeSpecName "kube-api-access-2wbkx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:36:51.658512 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.658369 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/773e5bdc-f53c-4109-bbc9-447290933152-kube-api-access-qgh6t" (OuterVolumeSpecName: "kube-api-access-qgh6t") pod "773e5bdc-f53c-4109-bbc9-447290933152" (UID: "773e5bdc-f53c-4109-bbc9-447290933152"). InnerVolumeSpecName "kube-api-access-qgh6t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:36:51.757506 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.757466 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54972101-69fe-4982-945d-4db862573444-plugins-config-volume\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:36:51.757506 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.757502 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2wbkx\" (UniqueName: \"kubernetes.io/projected/54972101-69fe-4982-945d-4db862573444-kube-api-access-2wbkx\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:36:51.757506 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.757512 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgh6t\" (UniqueName: \"kubernetes.io/projected/773e5bdc-f53c-4109-bbc9-447290933152-kube-api-access-qgh6t\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:36:51.837295 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.837202 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:51.957572 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.957538 2560 generic.go:356] "Generic (PLEG): container finished" podID="773e5bdc-f53c-4109-bbc9-447290933152" containerID="b6a40951850f269ae7045151c380420437ede3d81e2dc2f6db1878755b99f73b" exitCode=0 Sep 10 13:36:51.957759 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.957579 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-95ae6288-67867d978b-v5854" Sep 10 13:36:51.957759 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.957626 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-95ae6288-67867d978b-v5854" event={"ID":"773e5bdc-f53c-4109-bbc9-447290933152","Type":"ContainerDied","Data":"b6a40951850f269ae7045151c380420437ede3d81e2dc2f6db1878755b99f73b"} Sep 10 13:36:51.957759 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.957679 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-95ae6288-67867d978b-v5854" event={"ID":"773e5bdc-f53c-4109-bbc9-447290933152","Type":"ContainerDied","Data":"f4176cfd4c6b48730d45fd1adf4fcb4c93e17c9a7269969e02d95f0cbae587b3"} Sep 10 13:36:51.957759 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.957711 2560 scope.go:117] "RemoveContainer" containerID="b6a40951850f269ae7045151c380420437ede3d81e2dc2f6db1878755b99f73b" Sep 10 13:36:51.959098 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.959078 2560 generic.go:356] "Generic (PLEG): container finished" podID="54972101-69fe-4982-945d-4db862573444" containerID="538ee72895ef22a5f4b5c9973ea42c2b5298f7b411fa4364a6facec5fc1bf9ce" exitCode=0 Sep 10 13:36:51.959210 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.959111 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-95ae6288-79dd4886c-9cmns" event={"ID":"54972101-69fe-4982-945d-4db862573444","Type":"ContainerDied","Data":"538ee72895ef22a5f4b5c9973ea42c2b5298f7b411fa4364a6facec5fc1bf9ce"} Sep 10 13:36:51.959210 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.959132 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-95ae6288-79dd4886c-9cmns" Sep 10 13:36:51.959210 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.959145 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-95ae6288-79dd4886c-9cmns" event={"ID":"54972101-69fe-4982-945d-4db862573444","Type":"ContainerDied","Data":"3b0da71db48dbc4b3f0bb667b035b129387866093dc9948baac295a9ebd1b8e8"} Sep 10 13:36:51.969118 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.969098 2560 scope.go:117] "RemoveContainer" containerID="b6a40951850f269ae7045151c380420437ede3d81e2dc2f6db1878755b99f73b" Sep 10 13:36:51.969479 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:36:51.969458 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6a40951850f269ae7045151c380420437ede3d81e2dc2f6db1878755b99f73b\": container with ID starting with b6a40951850f269ae7045151c380420437ede3d81e2dc2f6db1878755b99f73b not found: ID does not exist" containerID="b6a40951850f269ae7045151c380420437ede3d81e2dc2f6db1878755b99f73b" Sep 10 13:36:51.969546 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.969488 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6a40951850f269ae7045151c380420437ede3d81e2dc2f6db1878755b99f73b"} err="failed to get container status \"b6a40951850f269ae7045151c380420437ede3d81e2dc2f6db1878755b99f73b\": rpc error: code = NotFound desc = could not find container \"b6a40951850f269ae7045151c380420437ede3d81e2dc2f6db1878755b99f73b\": container with ID starting with b6a40951850f269ae7045151c380420437ede3d81e2dc2f6db1878755b99f73b not found: ID does not exist" Sep 10 13:36:51.969546 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.969506 2560 scope.go:117] "RemoveContainer" containerID="538ee72895ef22a5f4b5c9973ea42c2b5298f7b411fa4364a6facec5fc1bf9ce" Sep 10 13:36:51.977983 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.977962 2560 scope.go:117] "RemoveContainer" containerID="538ee72895ef22a5f4b5c9973ea42c2b5298f7b411fa4364a6facec5fc1bf9ce" Sep 10 13:36:51.978265 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:36:51.978246 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"538ee72895ef22a5f4b5c9973ea42c2b5298f7b411fa4364a6facec5fc1bf9ce\": container with ID starting with 538ee72895ef22a5f4b5c9973ea42c2b5298f7b411fa4364a6facec5fc1bf9ce not found: ID does not exist" containerID="538ee72895ef22a5f4b5c9973ea42c2b5298f7b411fa4364a6facec5fc1bf9ce" Sep 10 13:36:51.978312 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.978273 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"538ee72895ef22a5f4b5c9973ea42c2b5298f7b411fa4364a6facec5fc1bf9ce"} err="failed to get container status \"538ee72895ef22a5f4b5c9973ea42c2b5298f7b411fa4364a6facec5fc1bf9ce\": rpc error: code = NotFound desc = could not find container \"538ee72895ef22a5f4b5c9973ea42c2b5298f7b411fa4364a6facec5fc1bf9ce\": container with ID starting with 538ee72895ef22a5f4b5c9973ea42c2b5298f7b411fa4364a6facec5fc1bf9ce not found: ID does not exist" Sep 10 13:36:51.988279 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.988253 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-95ae6288-67867d978b-v5854"] Sep 10 13:36:51.991039 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:51.991017 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-95ae6288-67867d978b-v5854"] Sep 10 13:36:52.002191 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:52.002163 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-95ae6288-79dd4886c-9cmns"] Sep 10 13:36:52.007330 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:52.007306 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-95ae6288-79dd4886c-9cmns"] Sep 10 13:36:52.563364 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:52.563281 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54972101-69fe-4982-945d-4db862573444" path="/var/lib/kubelet/pods/54972101-69fe-4982-945d-4db862573444/volumes" Sep 10 13:36:52.563754 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:52.563736 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="773e5bdc-f53c-4109-bbc9-447290933152" path="/var/lib/kubelet/pods/773e5bdc-f53c-4109-bbc9-447290933152/volumes" Sep 10 13:36:52.690776 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:52.690744 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2"] Sep 10 13:36:52.690986 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:52.690959 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" podUID="89ececc1-da3b-4ff4-a6cb-8cf608f0210b" containerName="istio-proxy" containerID="cri-o://ee693ea0895a2f3acf6879a158b752370a397ceb8af1fd77423cd5dc2a702c8a" gracePeriod=30 Sep 10 13:36:52.771150 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:52.771108 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:53.181345 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.181290 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:53.550522 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.550438 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz"] Sep 10 13:36:53.550871 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.550858 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54972101-69fe-4982-945d-4db862573444" containerName="payload-processing" Sep 10 13:36:53.550918 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.550873 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="54972101-69fe-4982-945d-4db862573444" containerName="payload-processing" Sep 10 13:36:53.550918 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.550891 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="773e5bdc-f53c-4109-bbc9-447290933152" containerName="payload-pre-processing" Sep 10 13:36:53.550918 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.550897 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="773e5bdc-f53c-4109-bbc9-447290933152" containerName="payload-pre-processing" Sep 10 13:36:53.551031 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.550967 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="54972101-69fe-4982-945d-4db862573444" containerName="payload-processing" Sep 10 13:36:53.551031 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.550976 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="773e5bdc-f53c-4109-bbc9-447290933152" containerName="payload-pre-processing" Sep 10 13:36:53.555634 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.555617 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:36:53.558525 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.558495 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-2e7a3258-openshift-default-dockercfg-8mmfl\"" Sep 10 13:36:53.572526 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.572498 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-istio-token\") pod \"e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz\" (UID: \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\") " pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:36:53.572977 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.572561 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-istio-envoy\") pod \"e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz\" (UID: \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\") " pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:36:53.572977 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.572590 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skmp7\" (UniqueName: \"kubernetes.io/projected/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-kube-api-access-skmp7\") pod \"e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz\" (UID: \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\") " pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:36:53.572977 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.572693 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-workload-certs\") pod \"e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz\" (UID: \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\") " pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:36:53.572977 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.572744 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-workload-socket\") pod \"e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz\" (UID: \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\") " pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:36:53.572977 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.572784 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-credential-socket\") pod \"e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz\" (UID: \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\") " pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:36:53.572977 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.572812 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-istio-data\") pod \"e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz\" (UID: \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\") " pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:36:53.572977 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.572837 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-istiod-ca-cert\") pod \"e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz\" (UID: \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\") " pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:36:53.572977 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.572872 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-istio-podinfo\") pod \"e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz\" (UID: \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\") " pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:36:53.575045 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.575020 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz"] Sep 10 13:36:53.648171 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.648137 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:53.674331 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.674284 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-istio-token\") pod \"e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz\" (UID: \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\") " pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:36:53.674592 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.674351 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-istio-envoy\") pod \"e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz\" (UID: \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\") " pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:36:53.674592 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.674371 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skmp7\" (UniqueName: \"kubernetes.io/projected/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-kube-api-access-skmp7\") pod \"e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz\" (UID: \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\") " pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:36:53.674592 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.674417 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-workload-certs\") pod \"e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz\" (UID: \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\") " pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:36:53.674592 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.674463 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-workload-socket\") pod \"e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz\" (UID: \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\") " pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:36:53.674592 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.674505 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-credential-socket\") pod \"e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz\" (UID: \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\") " pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:36:53.674592 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.674532 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-istio-data\") pod \"e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz\" (UID: \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\") " pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:36:53.674592 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.674556 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-istiod-ca-cert\") pod \"e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz\" (UID: \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\") " pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:36:53.674592 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.674581 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-istio-podinfo\") pod \"e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz\" (UID: \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\") " pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:36:53.675091 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.674929 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-workload-certs\") pod \"e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz\" (UID: \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\") " pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:36:53.675091 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.674962 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-workload-socket\") pod \"e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz\" (UID: \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\") " pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:36:53.675184 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.675141 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-istio-data\") pod \"e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz\" (UID: \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\") " pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:36:53.675184 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.675164 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-credential-socket\") pod \"e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz\" (UID: \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\") " pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:36:53.675359 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.675324 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-istiod-ca-cert\") pod \"e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz\" (UID: \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\") " pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:36:53.676996 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.676970 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-istio-envoy\") pod \"e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz\" (UID: \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\") " pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:36:53.677145 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.677125 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-istio-podinfo\") pod \"e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz\" (UID: \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\") " pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:36:53.692842 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.692798 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-istio-token\") pod \"e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz\" (UID: \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\") " pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:36:53.692988 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.692807 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skmp7\" (UniqueName: \"kubernetes.io/projected/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-kube-api-access-skmp7\") pod \"e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz\" (UID: \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\") " pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:36:53.768608 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.768571 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:53.866796 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:53.866694 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:36:54.155122 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:54.155044 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz"] Sep 10 13:36:54.158889 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:36:54.158860 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f04e35a_f79e_43cc_91e7_64fd99c5a17c.slice/crio-75106dd63af68da2528689472665908d0f3189d26b88eaf8c4c5e7f1bf90368c WatchSource:0}: Error finding container 75106dd63af68da2528689472665908d0f3189d26b88eaf8c4c5e7f1bf90368c: Status 404 returned error can't find the container with id 75106dd63af68da2528689472665908d0f3189d26b88eaf8c4c5e7f1bf90368c Sep 10 13:36:54.161219 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:54.161189 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:36:54.161300 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:54.161264 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:36:54.161300 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:54.161294 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:36:54.834134 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:54.834099 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:54.985991 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:54.985941 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" event={"ID":"9f04e35a-f79e-43cc-91e7-64fd99c5a17c","Type":"ContainerStarted","Data":"18bbbe991d866db01bc8fc7ed4b406f2e78f715e09fafb95d9b379a448a3a830"} Sep 10 13:36:54.986192 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:54.986042 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" event={"ID":"9f04e35a-f79e-43cc-91e7-64fd99c5a17c","Type":"ContainerStarted","Data":"75106dd63af68da2528689472665908d0f3189d26b88eaf8c4c5e7f1bf90368c"} Sep 10 13:36:55.014713 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:55.014655 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" podStartSLOduration=2.01463934 podStartE2EDuration="2.01463934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:36:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:36:55.011126645 +0000 UTC m=+1124.905844063" watchObservedRunningTime="2026-09-10 13:36:55.01463934 +0000 UTC m=+1124.909356827" Sep 10 13:36:55.867913 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:55.867878 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:36:55.873323 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:55.873295 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:36:55.992325 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:55.992288 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:36:55.993436 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:55.993392 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:36:56.454300 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:56.454266 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:56.582891 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:56.582851 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:56.898575 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:56.898479 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:57.872095 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:57.872067 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:57.958815 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:57.958754 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:58.007646 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.007611 2560 generic.go:356] "Generic (PLEG): container finished" podID="89ececc1-da3b-4ff4-a6cb-8cf608f0210b" containerID="ee693ea0895a2f3acf6879a158b752370a397ceb8af1fd77423cd5dc2a702c8a" exitCode=0 Sep 10 13:36:58.007792 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.007651 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" event={"ID":"89ececc1-da3b-4ff4-a6cb-8cf608f0210b","Type":"ContainerDied","Data":"ee693ea0895a2f3acf6879a158b752370a397ceb8af1fd77423cd5dc2a702c8a"} Sep 10 13:36:58.007792 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.007669 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" Sep 10 13:36:58.007792 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.007700 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2" event={"ID":"89ececc1-da3b-4ff4-a6cb-8cf608f0210b","Type":"ContainerDied","Data":"610ad242948498a6fc60f1a6ee3556a6553371bf3156a745a736622eacab64d3"} Sep 10 13:36:58.007792 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.007732 2560 scope.go:117] "RemoveContainer" containerID="ee693ea0895a2f3acf6879a158b752370a397ceb8af1fd77423cd5dc2a702c8a" Sep 10 13:36:58.017626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.017573 2560 scope.go:117] "RemoveContainer" containerID="d506f261a251ba5cf7a72a39fb4676edb17c592ca9088f58d38e44d5a69be456" Sep 10 13:36:58.021046 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.021022 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-credential-socket\") pod \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\" (UID: \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\") " Sep 10 13:36:58.021179 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.021063 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-workload-certs\") pod \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\" (UID: \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\") " Sep 10 13:36:58.021179 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.021093 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nksll\" (UniqueName: \"kubernetes.io/projected/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-kube-api-access-nksll\") pod \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\" (UID: \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\") " Sep 10 13:36:58.021179 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.021122 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-workload-socket\") pod \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\" (UID: \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\") " Sep 10 13:36:58.021179 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.021146 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-istio-data\") pod \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\" (UID: \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\") " Sep 10 13:36:58.021426 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.021188 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-istio-envoy\") pod \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\" (UID: \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\") " Sep 10 13:36:58.021426 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.021212 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-istiod-ca-cert\") pod \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\" (UID: \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\") " Sep 10 13:36:58.021426 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.021253 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-istio-token\") pod \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\" (UID: \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\") " Sep 10 13:36:58.021426 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.021305 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-istio-podinfo\") pod \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\" (UID: \"89ececc1-da3b-4ff4-a6cb-8cf608f0210b\") " Sep 10 13:36:58.021426 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.021326 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "89ececc1-da3b-4ff4-a6cb-8cf608f0210b" (UID: "89ececc1-da3b-4ff4-a6cb-8cf608f0210b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:36:58.021426 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.021342 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "89ececc1-da3b-4ff4-a6cb-8cf608f0210b" (UID: "89ececc1-da3b-4ff4-a6cb-8cf608f0210b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:36:58.021426 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.021355 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "89ececc1-da3b-4ff4-a6cb-8cf608f0210b" (UID: "89ececc1-da3b-4ff4-a6cb-8cf608f0210b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:36:58.021830 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.021540 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-istio-data" (OuterVolumeSpecName: "istio-data") pod "89ececc1-da3b-4ff4-a6cb-8cf608f0210b" (UID: "89ececc1-da3b-4ff4-a6cb-8cf608f0210b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:36:58.021830 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.021751 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "89ececc1-da3b-4ff4-a6cb-8cf608f0210b" (UID: "89ececc1-da3b-4ff4-a6cb-8cf608f0210b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:36:58.021908 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.021848 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-workload-socket\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:36:58.021908 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.021872 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-istio-data\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:36:58.021908 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.021886 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-credential-socket\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:36:58.021908 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.021900 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-workload-certs\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:36:58.024002 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.023974 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-kube-api-access-nksll" (OuterVolumeSpecName: "kube-api-access-nksll") pod "89ececc1-da3b-4ff4-a6cb-8cf608f0210b" (UID: "89ececc1-da3b-4ff4-a6cb-8cf608f0210b"). InnerVolumeSpecName "kube-api-access-nksll". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:36:58.024112 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.024019 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "89ececc1-da3b-4ff4-a6cb-8cf608f0210b" (UID: "89ececc1-da3b-4ff4-a6cb-8cf608f0210b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 13:36:58.024112 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.024037 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-istio-token" (OuterVolumeSpecName: "istio-token") pod "89ececc1-da3b-4ff4-a6cb-8cf608f0210b" (UID: "89ececc1-da3b-4ff4-a6cb-8cf608f0210b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:36:58.024300 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.024278 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "89ececc1-da3b-4ff4-a6cb-8cf608f0210b" (UID: "89ececc1-da3b-4ff4-a6cb-8cf608f0210b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:36:58.034328 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.034219 2560 scope.go:117] "RemoveContainer" containerID="ee693ea0895a2f3acf6879a158b752370a397ceb8af1fd77423cd5dc2a702c8a" Sep 10 13:36:58.034991 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:36:58.034957 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee693ea0895a2f3acf6879a158b752370a397ceb8af1fd77423cd5dc2a702c8a\": container with ID starting with ee693ea0895a2f3acf6879a158b752370a397ceb8af1fd77423cd5dc2a702c8a not found: ID does not exist" containerID="ee693ea0895a2f3acf6879a158b752370a397ceb8af1fd77423cd5dc2a702c8a" Sep 10 13:36:58.035208 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.035010 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee693ea0895a2f3acf6879a158b752370a397ceb8af1fd77423cd5dc2a702c8a"} err="failed to get container status \"ee693ea0895a2f3acf6879a158b752370a397ceb8af1fd77423cd5dc2a702c8a\": rpc error: code = NotFound desc = could not find container \"ee693ea0895a2f3acf6879a158b752370a397ceb8af1fd77423cd5dc2a702c8a\": container with ID starting with ee693ea0895a2f3acf6879a158b752370a397ceb8af1fd77423cd5dc2a702c8a not found: ID does not exist" Sep 10 13:36:58.035208 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.035036 2560 scope.go:117] "RemoveContainer" containerID="d506f261a251ba5cf7a72a39fb4676edb17c592ca9088f58d38e44d5a69be456" Sep 10 13:36:58.035434 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:36:58.035391 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d506f261a251ba5cf7a72a39fb4676edb17c592ca9088f58d38e44d5a69be456\": container with ID starting with d506f261a251ba5cf7a72a39fb4676edb17c592ca9088f58d38e44d5a69be456 not found: ID does not exist" containerID="d506f261a251ba5cf7a72a39fb4676edb17c592ca9088f58d38e44d5a69be456" Sep 10 13:36:58.035544 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.035442 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d506f261a251ba5cf7a72a39fb4676edb17c592ca9088f58d38e44d5a69be456"} err="failed to get container status \"d506f261a251ba5cf7a72a39fb4676edb17c592ca9088f58d38e44d5a69be456\": rpc error: code = NotFound desc = could not find container \"d506f261a251ba5cf7a72a39fb4676edb17c592ca9088f58d38e44d5a69be456\": container with ID starting with d506f261a251ba5cf7a72a39fb4676edb17c592ca9088f58d38e44d5a69be456 not found: ID does not exist" Sep 10 13:36:58.041837 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.041781 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:58.122673 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.122640 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-istio-envoy\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:36:58.122673 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.122671 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-istiod-ca-cert\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:36:58.122673 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.122679 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-istio-token\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:36:58.122917 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.122687 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-istio-podinfo\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:36:58.122917 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.122697 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nksll\" (UniqueName: \"kubernetes.io/projected/89ececc1-da3b-4ff4-a6cb-8cf608f0210b-kube-api-access-nksll\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:36:58.341690 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.341651 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2"] Sep 10 13:36:58.346852 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.346819 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-95ae6288-openshift-default-c5f5584d4-x9qc2"] Sep 10 13:36:58.509487 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.509450 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:58.563195 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.563164 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89ececc1-da3b-4ff4-a6cb-8cf608f0210b" path="/var/lib/kubelet/pods/89ececc1-da3b-4ff4-a6cb-8cf608f0210b/volumes" Sep 10 13:36:58.619677 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.619590 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:36:58.708998 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.708963 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2e7a3258-566fd9cd46-8rfxb"] Sep 10 13:36:58.709396 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.709382 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89ececc1-da3b-4ff4-a6cb-8cf608f0210b" containerName="istio-proxy" Sep 10 13:36:58.709396 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.709397 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="89ececc1-da3b-4ff4-a6cb-8cf608f0210b" containerName="istio-proxy" Sep 10 13:36:58.709515 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.709503 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="89ececc1-da3b-4ff4-a6cb-8cf608f0210b" containerName="istio-proxy" Sep 10 13:36:58.709515 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.709511 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="89ececc1-da3b-4ff4-a6cb-8cf608f0210b" containerName="istio-proxy" Sep 10 13:36:58.714228 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.714205 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-2e7a3258-566fd9cd46-8rfxb" Sep 10 13:36:58.716804 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.716774 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-2e7a3258-dockercfg-b4pk6\"" Sep 10 13:36:58.716933 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.716840 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-2e7a3258\"" Sep 10 13:36:58.735075 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.735045 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2e7a3258-566fd9cd46-8rfxb"] Sep 10 13:36:58.821944 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.821910 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2e7a3258-5f55849885-gwmg9"] Sep 10 13:36:58.822356 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.822344 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89ececc1-da3b-4ff4-a6cb-8cf608f0210b" containerName="istio-proxy" Sep 10 13:36:58.822426 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.822358 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="89ececc1-da3b-4ff4-a6cb-8cf608f0210b" containerName="istio-proxy" Sep 10 13:36:58.825658 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.825636 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-2e7a3258-5f55849885-gwmg9" Sep 10 13:36:58.829565 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.829541 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcl2k\" (UniqueName: \"kubernetes.io/projected/3301c0ac-5c0e-47ea-8ba3-d4d2e0065c93-kube-api-access-kcl2k\") pod \"payload-pre-processing-e2e-derive-2e7a3258-566fd9cd46-8rfxb\" (UID: \"3301c0ac-5c0e-47ea-8ba3-d4d2e0065c93\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2e7a3258-566fd9cd46-8rfxb" Sep 10 13:36:58.829666 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.829598 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3301c0ac-5c0e-47ea-8ba3-d4d2e0065c93-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-2e7a3258-566fd9cd46-8rfxb\" (UID: \"3301c0ac-5c0e-47ea-8ba3-d4d2e0065c93\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2e7a3258-566fd9cd46-8rfxb" Sep 10 13:36:58.845253 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.845217 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2e7a3258-5f55849885-gwmg9"] Sep 10 13:36:58.932152 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.932101 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcl2k\" (UniqueName: \"kubernetes.io/projected/3301c0ac-5c0e-47ea-8ba3-d4d2e0065c93-kube-api-access-kcl2k\") pod \"payload-pre-processing-e2e-derive-2e7a3258-566fd9cd46-8rfxb\" (UID: \"3301c0ac-5c0e-47ea-8ba3-d4d2e0065c93\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2e7a3258-566fd9cd46-8rfxb" Sep 10 13:36:58.932152 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.932225 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3301c0ac-5c0e-47ea-8ba3-d4d2e0065c93-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-2e7a3258-566fd9cd46-8rfxb\" (UID: \"3301c0ac-5c0e-47ea-8ba3-d4d2e0065c93\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2e7a3258-566fd9cd46-8rfxb" Sep 10 13:36:58.932152 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.932314 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97b7f072-3c7d-4241-838e-8baef91ae711-plugins-config-volume\") pod \"payload-processing-e2e-derive-2e7a3258-5f55849885-gwmg9\" (UID: \"97b7f072-3c7d-4241-838e-8baef91ae711\") " pod="openshift-ingress/payload-processing-e2e-derive-2e7a3258-5f55849885-gwmg9" Sep 10 13:36:58.932152 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.932365 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktvtg\" (UniqueName: \"kubernetes.io/projected/97b7f072-3c7d-4241-838e-8baef91ae711-kube-api-access-ktvtg\") pod \"payload-processing-e2e-derive-2e7a3258-5f55849885-gwmg9\" (UID: \"97b7f072-3c7d-4241-838e-8baef91ae711\") " pod="openshift-ingress/payload-processing-e2e-derive-2e7a3258-5f55849885-gwmg9" Sep 10 13:36:58.933361 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.933335 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3301c0ac-5c0e-47ea-8ba3-d4d2e0065c93-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-2e7a3258-566fd9cd46-8rfxb\" (UID: \"3301c0ac-5c0e-47ea-8ba3-d4d2e0065c93\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2e7a3258-566fd9cd46-8rfxb" Sep 10 13:36:58.942965 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:58.942934 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcl2k\" (UniqueName: \"kubernetes.io/projected/3301c0ac-5c0e-47ea-8ba3-d4d2e0065c93-kube-api-access-kcl2k\") pod \"payload-pre-processing-e2e-derive-2e7a3258-566fd9cd46-8rfxb\" (UID: \"3301c0ac-5c0e-47ea-8ba3-d4d2e0065c93\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2e7a3258-566fd9cd46-8rfxb" Sep 10 13:36:59.024802 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:59.024768 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-2e7a3258-566fd9cd46-8rfxb" Sep 10 13:36:59.033140 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:59.033102 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97b7f072-3c7d-4241-838e-8baef91ae711-plugins-config-volume\") pod \"payload-processing-e2e-derive-2e7a3258-5f55849885-gwmg9\" (UID: \"97b7f072-3c7d-4241-838e-8baef91ae711\") " pod="openshift-ingress/payload-processing-e2e-derive-2e7a3258-5f55849885-gwmg9" Sep 10 13:36:59.033287 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:59.033142 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktvtg\" (UniqueName: \"kubernetes.io/projected/97b7f072-3c7d-4241-838e-8baef91ae711-kube-api-access-ktvtg\") pod \"payload-processing-e2e-derive-2e7a3258-5f55849885-gwmg9\" (UID: \"97b7f072-3c7d-4241-838e-8baef91ae711\") " pod="openshift-ingress/payload-processing-e2e-derive-2e7a3258-5f55849885-gwmg9" Sep 10 13:36:59.033855 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:59.033833 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97b7f072-3c7d-4241-838e-8baef91ae711-plugins-config-volume\") pod \"payload-processing-e2e-derive-2e7a3258-5f55849885-gwmg9\" (UID: \"97b7f072-3c7d-4241-838e-8baef91ae711\") " pod="openshift-ingress/payload-processing-e2e-derive-2e7a3258-5f55849885-gwmg9" Sep 10 13:36:59.051535 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:59.051508 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktvtg\" (UniqueName: \"kubernetes.io/projected/97b7f072-3c7d-4241-838e-8baef91ae711-kube-api-access-ktvtg\") pod \"payload-processing-e2e-derive-2e7a3258-5f55849885-gwmg9\" (UID: \"97b7f072-3c7d-4241-838e-8baef91ae711\") " pod="openshift-ingress/payload-processing-e2e-derive-2e7a3258-5f55849885-gwmg9" Sep 10 13:36:59.136555 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:59.136508 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-2e7a3258-5f55849885-gwmg9" Sep 10 13:36:59.308650 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:59.308613 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2e7a3258-566fd9cd46-8rfxb"] Sep 10 13:36:59.417827 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:36:59.417793 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2e7a3258-5f55849885-gwmg9"] Sep 10 13:36:59.418980 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:36:59.418951 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97b7f072_3c7d_4241_838e_8baef91ae711.slice/crio-fe1e6479a05cacb1ea5f1890ecf55abb6467654a809b5552b256c5693b1236c1 WatchSource:0}: Error finding container fe1e6479a05cacb1ea5f1890ecf55abb6467654a809b5552b256c5693b1236c1: Status 404 returned error can't find the container with id fe1e6479a05cacb1ea5f1890ecf55abb6467654a809b5552b256c5693b1236c1 Sep 10 13:37:00.022746 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:00.022707 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-2e7a3258-566fd9cd46-8rfxb" event={"ID":"3301c0ac-5c0e-47ea-8ba3-d4d2e0065c93","Type":"ContainerStarted","Data":"c02ae77e839988d7909c5171eb4579b82fc57442eb11d6010226fa6efab6d9ce"} Sep 10 13:37:00.022954 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:00.022766 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-2e7a3258-566fd9cd46-8rfxb" event={"ID":"3301c0ac-5c0e-47ea-8ba3-d4d2e0065c93","Type":"ContainerStarted","Data":"5cf85b424308f33161c9c1f822823f74ab1b3c8e723be91524d03c8dd9eebbc7"} Sep 10 13:37:00.022954 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:00.022800 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-2e7a3258-566fd9cd46-8rfxb" Sep 10 13:37:00.024793 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:00.024760 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-2e7a3258-5f55849885-gwmg9" event={"ID":"97b7f072-3c7d-4241-838e-8baef91ae711","Type":"ContainerStarted","Data":"7376baa89e4f7baf4753a5e62eabcb016c43581006be29c7221d504fc57f8f92"} Sep 10 13:37:00.024914 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:00.024818 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-2e7a3258-5f55849885-gwmg9" event={"ID":"97b7f072-3c7d-4241-838e-8baef91ae711","Type":"ContainerStarted","Data":"fe1e6479a05cacb1ea5f1890ecf55abb6467654a809b5552b256c5693b1236c1"} Sep 10 13:37:00.024914 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:00.024848 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-2e7a3258-5f55849885-gwmg9" Sep 10 13:37:00.060934 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:00.060876 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-2e7a3258-566fd9cd46-8rfxb" podStartSLOduration=2.060859691 podStartE2EDuration="2.060859691s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:36:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:37:00.058303867 +0000 UTC m=+1129.953021276" watchObservedRunningTime="2026-09-10 13:37:00.060859691 +0000 UTC m=+1129.955577108" Sep 10 13:37:11.033903 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:11.033871 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-2e7a3258-5f55849885-gwmg9" Sep 10 13:37:11.034325 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:11.033950 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-2e7a3258-566fd9cd46-8rfxb" Sep 10 13:37:11.057922 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:11.057852 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-2e7a3258-5f55849885-gwmg9" podStartSLOduration=13.057831115 podStartE2EDuration="13.057831115s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:36:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:37:00.09521254 +0000 UTC m=+1129.989929957" watchObservedRunningTime="2026-09-10 13:37:11.057831115 +0000 UTC m=+1140.952548533" Sep 10 13:37:15.127324 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:15.127233 2560 generic.go:356] "Generic (PLEG): container finished" podID="9f04e35a-f79e-43cc-91e7-64fd99c5a17c" containerID="18bbbe991d866db01bc8fc7ed4b406f2e78f715e09fafb95d9b379a448a3a830" exitCode=0 Sep 10 13:37:15.127324 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:15.127307 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" event={"ID":"9f04e35a-f79e-43cc-91e7-64fd99c5a17c","Type":"ContainerDied","Data":"18bbbe991d866db01bc8fc7ed4b406f2e78f715e09fafb95d9b379a448a3a830"} Sep 10 13:37:15.127813 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:15.127724 2560 scope.go:117] "RemoveContainer" containerID="18bbbe991d866db01bc8fc7ed4b406f2e78f715e09fafb95d9b379a448a3a830" Sep 10 13:37:15.128847 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:15.128822 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:37:15.128904 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:15.128883 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:37:15.128946 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:15.128914 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:37:16.135745 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:16.135706 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" event={"ID":"9f04e35a-f79e-43cc-91e7-64fd99c5a17c","Type":"ContainerStarted","Data":"c85b2d27870c85edba5c26170368a73d7a1627cd9bfccd43a3320d2ffc1968a2"} Sep 10 13:37:16.867885 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:16.867847 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:37:16.868248 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:16.868223 2560 patch_prober.go:28] interesting pod/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Sep 10 13:37:16.868450 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:16.868392 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" podUID="9f04e35a-f79e-43cc-91e7-64fd99c5a17c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Sep 10 13:37:17.833225 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:17.833192 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:37:17.869580 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:17.869510 2560 patch_prober.go:28] interesting pod/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Sep 10 13:37:17.869580 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:17.869577 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" podUID="9f04e35a-f79e-43cc-91e7-64fd99c5a17c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Sep 10 13:37:18.867213 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:18.867172 2560 patch_prober.go:28] interesting pod/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Sep 10 13:37:18.867867 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:18.867830 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" podUID="9f04e35a-f79e-43cc-91e7-64fd99c5a17c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Sep 10 13:37:19.867818 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:19.867777 2560 patch_prober.go:28] interesting pod/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Sep 10 13:37:19.868518 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:19.868478 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" podUID="9f04e35a-f79e-43cc-91e7-64fd99c5a17c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Sep 10 13:37:20.867220 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:20.867171 2560 patch_prober.go:28] interesting pod/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Sep 10 13:37:20.867463 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:20.867253 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" podUID="9f04e35a-f79e-43cc-91e7-64fd99c5a17c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Sep 10 13:37:21.871721 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:21.871693 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:37:21.872168 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:21.872148 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:37:21.872890 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:21.872870 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:37:22.715770 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:22.715739 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:37:26.212732 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:26.212670 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:37:26.559768 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:26.559681 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:37:27.946332 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:27.946285 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:37:27.977670 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:27.977632 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2e7a3258-5f55849885-gwmg9"] Sep 10 13:37:27.978133 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:27.978097 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-2e7a3258-5f55849885-gwmg9" podUID="97b7f072-3c7d-4241-838e-8baef91ae711" containerName="payload-processing" containerID="cri-o://7376baa89e4f7baf4753a5e62eabcb016c43581006be29c7221d504fc57f8f92" gracePeriod=30 Sep 10 13:37:27.988723 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:27.988679 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2e7a3258-566fd9cd46-8rfxb"] Sep 10 13:37:27.988994 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:27.988935 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-2e7a3258-566fd9cd46-8rfxb" podUID="3301c0ac-5c0e-47ea-8ba3-d4d2e0065c93" containerName="payload-pre-processing" containerID="cri-o://c02ae77e839988d7909c5171eb4579b82fc57442eb11d6010226fa6efab6d9ce" gracePeriod=30 Sep 10 13:37:28.046712 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:28.046680 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:37:28.146827 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:28.146787 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:37:28.223184 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:28.223083 2560 generic.go:356] "Generic (PLEG): container finished" podID="3301c0ac-5c0e-47ea-8ba3-d4d2e0065c93" containerID="c02ae77e839988d7909c5171eb4579b82fc57442eb11d6010226fa6efab6d9ce" exitCode=0 Sep 10 13:37:28.223184 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:28.223161 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-2e7a3258-566fd9cd46-8rfxb" event={"ID":"3301c0ac-5c0e-47ea-8ba3-d4d2e0065c93","Type":"ContainerDied","Data":"c02ae77e839988d7909c5171eb4579b82fc57442eb11d6010226fa6efab6d9ce"} Sep 10 13:37:28.225070 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:28.225044 2560 generic.go:356] "Generic (PLEG): container finished" podID="97b7f072-3c7d-4241-838e-8baef91ae711" containerID="7376baa89e4f7baf4753a5e62eabcb016c43581006be29c7221d504fc57f8f92" exitCode=0 Sep 10 13:37:28.225202 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:28.225091 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-2e7a3258-5f55849885-gwmg9" event={"ID":"97b7f072-3c7d-4241-838e-8baef91ae711","Type":"ContainerDied","Data":"7376baa89e4f7baf4753a5e62eabcb016c43581006be29c7221d504fc57f8f92"} Sep 10 13:37:28.246134 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:28.246100 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:37:28.267075 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:28.267048 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-2e7a3258-5f55849885-gwmg9" Sep 10 13:37:28.269930 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:28.269909 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-2e7a3258-566fd9cd46-8rfxb" Sep 10 13:37:28.345371 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:28.345338 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz"] Sep 10 13:37:28.345602 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:28.345568 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" podUID="9f04e35a-f79e-43cc-91e7-64fd99c5a17c" containerName="istio-proxy" containerID="cri-o://c85b2d27870c85edba5c26170368a73d7a1627cd9bfccd43a3320d2ffc1968a2" gracePeriod=30 Sep 10 13:37:28.356189 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:28.356160 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kcl2k\" (UniqueName: \"kubernetes.io/projected/3301c0ac-5c0e-47ea-8ba3-d4d2e0065c93-kube-api-access-kcl2k\") pod \"3301c0ac-5c0e-47ea-8ba3-d4d2e0065c93\" (UID: \"3301c0ac-5c0e-47ea-8ba3-d4d2e0065c93\") " Sep 10 13:37:28.356347 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:28.356261 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97b7f072-3c7d-4241-838e-8baef91ae711-plugins-config-volume\") pod \"97b7f072-3c7d-4241-838e-8baef91ae711\" (UID: \"97b7f072-3c7d-4241-838e-8baef91ae711\") " Sep 10 13:37:28.356347 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:28.356299 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ktvtg\" (UniqueName: \"kubernetes.io/projected/97b7f072-3c7d-4241-838e-8baef91ae711-kube-api-access-ktvtg\") pod \"97b7f072-3c7d-4241-838e-8baef91ae711\" (UID: \"97b7f072-3c7d-4241-838e-8baef91ae711\") " Sep 10 13:37:28.356456 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:28.356344 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3301c0ac-5c0e-47ea-8ba3-d4d2e0065c93-plugins-config-volume\") pod \"3301c0ac-5c0e-47ea-8ba3-d4d2e0065c93\" (UID: \"3301c0ac-5c0e-47ea-8ba3-d4d2e0065c93\") " Sep 10 13:37:28.356780 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:28.356747 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97b7f072-3c7d-4241-838e-8baef91ae711-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "97b7f072-3c7d-4241-838e-8baef91ae711" (UID: "97b7f072-3c7d-4241-838e-8baef91ae711"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:37:28.356892 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:28.356805 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3301c0ac-5c0e-47ea-8ba3-d4d2e0065c93-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3301c0ac-5c0e-47ea-8ba3-d4d2e0065c93" (UID: "3301c0ac-5c0e-47ea-8ba3-d4d2e0065c93"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:37:28.358297 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:28.358278 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97b7f072-3c7d-4241-838e-8baef91ae711-kube-api-access-ktvtg" (OuterVolumeSpecName: "kube-api-access-ktvtg") pod "97b7f072-3c7d-4241-838e-8baef91ae711" (UID: "97b7f072-3c7d-4241-838e-8baef91ae711"). InnerVolumeSpecName "kube-api-access-ktvtg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:37:28.358414 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:28.358383 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3301c0ac-5c0e-47ea-8ba3-d4d2e0065c93-kube-api-access-kcl2k" (OuterVolumeSpecName: "kube-api-access-kcl2k") pod "3301c0ac-5c0e-47ea-8ba3-d4d2e0065c93" (UID: "3301c0ac-5c0e-47ea-8ba3-d4d2e0065c93"). InnerVolumeSpecName "kube-api-access-kcl2k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:37:28.361826 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:28.361802 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:37:28.457783 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:28.457748 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kcl2k\" (UniqueName: \"kubernetes.io/projected/3301c0ac-5c0e-47ea-8ba3-d4d2e0065c93-kube-api-access-kcl2k\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:37:28.457783 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:28.457777 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97b7f072-3c7d-4241-838e-8baef91ae711-plugins-config-volume\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:37:28.457783 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:28.457787 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ktvtg\" (UniqueName: \"kubernetes.io/projected/97b7f072-3c7d-4241-838e-8baef91ae711-kube-api-access-ktvtg\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:37:28.458031 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:28.457798 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3301c0ac-5c0e-47ea-8ba3-d4d2e0065c93-plugins-config-volume\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:37:29.068412 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:29.068372 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:37:29.199141 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:29.199105 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:37:29.233380 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:29.233356 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-2e7a3258-566fd9cd46-8rfxb" Sep 10 13:37:29.233577 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:29.233387 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-2e7a3258-566fd9cd46-8rfxb" event={"ID":"3301c0ac-5c0e-47ea-8ba3-d4d2e0065c93","Type":"ContainerDied","Data":"5cf85b424308f33161c9c1f822823f74ab1b3c8e723be91524d03c8dd9eebbc7"} Sep 10 13:37:29.233577 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:29.233485 2560 scope.go:117] "RemoveContainer" containerID="c02ae77e839988d7909c5171eb4579b82fc57442eb11d6010226fa6efab6d9ce" Sep 10 13:37:29.234878 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:29.234860 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-2e7a3258-5f55849885-gwmg9" event={"ID":"97b7f072-3c7d-4241-838e-8baef91ae711","Type":"ContainerDied","Data":"fe1e6479a05cacb1ea5f1890ecf55abb6467654a809b5552b256c5693b1236c1"} Sep 10 13:37:29.234961 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:29.234896 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-2e7a3258-5f55849885-gwmg9" Sep 10 13:37:29.242921 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:29.242903 2560 scope.go:117] "RemoveContainer" containerID="7376baa89e4f7baf4753a5e62eabcb016c43581006be29c7221d504fc57f8f92" Sep 10 13:37:29.264908 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:29.264880 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2e7a3258-566fd9cd46-8rfxb"] Sep 10 13:37:29.272500 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:29.272474 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2e7a3258-566fd9cd46-8rfxb"] Sep 10 13:37:29.284833 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:29.284803 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2e7a3258-5f55849885-gwmg9"] Sep 10 13:37:29.295125 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:29.295093 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2e7a3258-5f55849885-gwmg9"] Sep 10 13:37:29.313259 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:29.313229 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:37:30.563093 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:30.563062 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3301c0ac-5c0e-47ea-8ba3-d4d2e0065c93" path="/var/lib/kubelet/pods/3301c0ac-5c0e-47ea-8ba3-d4d2e0065c93/volumes" Sep 10 13:37:30.563493 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:30.563477 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97b7f072-3c7d-4241-838e-8baef91ae711" path="/var/lib/kubelet/pods/97b7f072-3c7d-4241-838e-8baef91ae711/volumes" Sep 10 13:37:30.689943 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:30.689907 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:37:33.607740 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:33.607712 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:37:33.709189 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:33.709147 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-istio-envoy\") pod \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\" (UID: \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\") " Sep 10 13:37:33.709378 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:33.709202 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-credential-socket\") pod \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\" (UID: \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\") " Sep 10 13:37:33.709378 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:33.709250 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-istio-podinfo\") pod \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\" (UID: \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\") " Sep 10 13:37:33.709378 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:33.709292 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-workload-socket\") pod \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\" (UID: \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\") " Sep 10 13:37:33.709378 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:33.709368 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-workload-certs\") pod \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\" (UID: \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\") " Sep 10 13:37:33.709615 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:33.709397 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-istiod-ca-cert\") pod \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\" (UID: \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\") " Sep 10 13:37:33.709615 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:33.709453 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-skmp7\" (UniqueName: \"kubernetes.io/projected/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-kube-api-access-skmp7\") pod \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\" (UID: \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\") " Sep 10 13:37:33.709615 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:33.709513 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-istio-token\") pod \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\" (UID: \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\") " Sep 10 13:37:33.709615 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:33.709573 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-istio-data\") pod \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\" (UID: \"9f04e35a-f79e-43cc-91e7-64fd99c5a17c\") " Sep 10 13:37:33.709615 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:33.709587 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9f04e35a-f79e-43cc-91e7-64fd99c5a17c" (UID: "9f04e35a-f79e-43cc-91e7-64fd99c5a17c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:37:33.709919 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:33.709660 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9f04e35a-f79e-43cc-91e7-64fd99c5a17c" (UID: "9f04e35a-f79e-43cc-91e7-64fd99c5a17c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:37:33.709919 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:33.709762 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9f04e35a-f79e-43cc-91e7-64fd99c5a17c" (UID: "9f04e35a-f79e-43cc-91e7-64fd99c5a17c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:37:33.710025 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:33.709898 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-istio-data" (OuterVolumeSpecName: "istio-data") pod "9f04e35a-f79e-43cc-91e7-64fd99c5a17c" (UID: "9f04e35a-f79e-43cc-91e7-64fd99c5a17c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:37:33.710025 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:33.710008 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-istio-data\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:37:33.710025 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:33.710023 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-credential-socket\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:37:33.710143 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:33.710036 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-workload-socket\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:37:33.710143 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:33.710048 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-workload-certs\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:37:33.710143 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:33.710057 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9f04e35a-f79e-43cc-91e7-64fd99c5a17c" (UID: "9f04e35a-f79e-43cc-91e7-64fd99c5a17c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:37:33.711571 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:33.711549 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9f04e35a-f79e-43cc-91e7-64fd99c5a17c" (UID: "9f04e35a-f79e-43cc-91e7-64fd99c5a17c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:37:33.711683 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:33.711623 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-kube-api-access-skmp7" (OuterVolumeSpecName: "kube-api-access-skmp7") pod "9f04e35a-f79e-43cc-91e7-64fd99c5a17c" (UID: "9f04e35a-f79e-43cc-91e7-64fd99c5a17c"). InnerVolumeSpecName "kube-api-access-skmp7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:37:33.711883 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:33.711866 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-istio-token" (OuterVolumeSpecName: "istio-token") pod "9f04e35a-f79e-43cc-91e7-64fd99c5a17c" (UID: "9f04e35a-f79e-43cc-91e7-64fd99c5a17c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:37:33.711943 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:33.711866 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9f04e35a-f79e-43cc-91e7-64fd99c5a17c" (UID: "9f04e35a-f79e-43cc-91e7-64fd99c5a17c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 13:37:33.811488 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:33.811448 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-istio-token\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:37:33.811488 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:33.811480 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-istio-envoy\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:37:33.811488 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:33.811498 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-istio-podinfo\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:37:33.811751 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:33.811506 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-istiod-ca-cert\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:37:33.811751 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:33.811516 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-skmp7\" (UniqueName: \"kubernetes.io/projected/9f04e35a-f79e-43cc-91e7-64fd99c5a17c-kube-api-access-skmp7\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:37:34.270192 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:34.270156 2560 generic.go:356] "Generic (PLEG): container finished" podID="9f04e35a-f79e-43cc-91e7-64fd99c5a17c" containerID="c85b2d27870c85edba5c26170368a73d7a1627cd9bfccd43a3320d2ffc1968a2" exitCode=0 Sep 10 13:37:34.270372 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:34.270211 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" Sep 10 13:37:34.270372 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:34.270247 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" event={"ID":"9f04e35a-f79e-43cc-91e7-64fd99c5a17c","Type":"ContainerDied","Data":"c85b2d27870c85edba5c26170368a73d7a1627cd9bfccd43a3320d2ffc1968a2"} Sep 10 13:37:34.270372 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:34.270307 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz" event={"ID":"9f04e35a-f79e-43cc-91e7-64fd99c5a17c","Type":"ContainerDied","Data":"75106dd63af68da2528689472665908d0f3189d26b88eaf8c4c5e7f1bf90368c"} Sep 10 13:37:34.270372 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:34.270332 2560 scope.go:117] "RemoveContainer" containerID="c85b2d27870c85edba5c26170368a73d7a1627cd9bfccd43a3320d2ffc1968a2" Sep 10 13:37:34.280830 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:34.280812 2560 scope.go:117] "RemoveContainer" containerID="18bbbe991d866db01bc8fc7ed4b406f2e78f715e09fafb95d9b379a448a3a830" Sep 10 13:37:34.289516 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:34.289490 2560 scope.go:117] "RemoveContainer" containerID="c85b2d27870c85edba5c26170368a73d7a1627cd9bfccd43a3320d2ffc1968a2" Sep 10 13:37:34.289824 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:37:34.289806 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c85b2d27870c85edba5c26170368a73d7a1627cd9bfccd43a3320d2ffc1968a2\": container with ID starting with c85b2d27870c85edba5c26170368a73d7a1627cd9bfccd43a3320d2ffc1968a2 not found: ID does not exist" containerID="c85b2d27870c85edba5c26170368a73d7a1627cd9bfccd43a3320d2ffc1968a2" Sep 10 13:37:34.289870 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:34.289835 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c85b2d27870c85edba5c26170368a73d7a1627cd9bfccd43a3320d2ffc1968a2"} err="failed to get container status \"c85b2d27870c85edba5c26170368a73d7a1627cd9bfccd43a3320d2ffc1968a2\": rpc error: code = NotFound desc = could not find container \"c85b2d27870c85edba5c26170368a73d7a1627cd9bfccd43a3320d2ffc1968a2\": container with ID starting with c85b2d27870c85edba5c26170368a73d7a1627cd9bfccd43a3320d2ffc1968a2 not found: ID does not exist" Sep 10 13:37:34.289870 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:34.289854 2560 scope.go:117] "RemoveContainer" containerID="18bbbe991d866db01bc8fc7ed4b406f2e78f715e09fafb95d9b379a448a3a830" Sep 10 13:37:34.290140 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:37:34.290110 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18bbbe991d866db01bc8fc7ed4b406f2e78f715e09fafb95d9b379a448a3a830\": container with ID starting with 18bbbe991d866db01bc8fc7ed4b406f2e78f715e09fafb95d9b379a448a3a830 not found: ID does not exist" containerID="18bbbe991d866db01bc8fc7ed4b406f2e78f715e09fafb95d9b379a448a3a830" Sep 10 13:37:34.290204 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:34.290146 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18bbbe991d866db01bc8fc7ed4b406f2e78f715e09fafb95d9b379a448a3a830"} err="failed to get container status \"18bbbe991d866db01bc8fc7ed4b406f2e78f715e09fafb95d9b379a448a3a830\": rpc error: code = NotFound desc = could not find container \"18bbbe991d866db01bc8fc7ed4b406f2e78f715e09fafb95d9b379a448a3a830\": container with ID starting with 18bbbe991d866db01bc8fc7ed4b406f2e78f715e09fafb95d9b379a448a3a830 not found: ID does not exist" Sep 10 13:37:34.296520 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:34.296494 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz"] Sep 10 13:37:34.300965 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:34.300937 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-2e7a3258-openshift-default-96d9cfcfb-lrwbz"] Sep 10 13:37:34.562882 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:34.562803 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f04e35a-f79e-43cc-91e7-64fd99c5a17c" path="/var/lib/kubelet/pods/9f04e35a-f79e-43cc-91e7-64fd99c5a17c/volumes" Sep 10 13:37:37.152707 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:37.152668 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:37:37.193841 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:37.193803 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-3c32b1-86f4465b45-s8jd6"] Sep 10 13:37:37.194104 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:37.194071 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-3c32b1-86f4465b45-s8jd6" podUID="f193a6dd-401a-4763-bc0c-3ee20a960046" containerName="payload-processing" containerID="cri-o://5a4cbae318d9a5e9cb6dd68f05628239a493aba4244e045c642d14e7ea5cda40" gracePeriod=30 Sep 10 13:37:37.246771 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:37.246733 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:37:37.345034 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:37.344956 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:37:37.446037 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:37.446004 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:37:37.459271 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:37.459245 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-3c32b1-86f4465b45-s8jd6" Sep 10 13:37:37.541910 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:37.541871 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:37:37.550002 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:37.549969 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f193a6dd-401a-4763-bc0c-3ee20a960046-plugins-config-volume\") pod \"f193a6dd-401a-4763-bc0c-3ee20a960046\" (UID: \"f193a6dd-401a-4763-bc0c-3ee20a960046\") " Sep 10 13:37:37.550153 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:37.550053 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pr7hb\" (UniqueName: \"kubernetes.io/projected/f193a6dd-401a-4763-bc0c-3ee20a960046-kube-api-access-pr7hb\") pod \"f193a6dd-401a-4763-bc0c-3ee20a960046\" (UID: \"f193a6dd-401a-4763-bc0c-3ee20a960046\") " Sep 10 13:37:37.550343 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:37.550320 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f193a6dd-401a-4763-bc0c-3ee20a960046-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f193a6dd-401a-4763-bc0c-3ee20a960046" (UID: "f193a6dd-401a-4763-bc0c-3ee20a960046"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:37:37.552215 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:37.552193 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f193a6dd-401a-4763-bc0c-3ee20a960046-kube-api-access-pr7hb" (OuterVolumeSpecName: "kube-api-access-pr7hb") pod "f193a6dd-401a-4763-bc0c-3ee20a960046" (UID: "f193a6dd-401a-4763-bc0c-3ee20a960046"). InnerVolumeSpecName "kube-api-access-pr7hb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:37:37.650910 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:37.650874 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f193a6dd-401a-4763-bc0c-3ee20a960046-plugins-config-volume\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:37:37.650910 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:37.650904 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pr7hb\" (UniqueName: \"kubernetes.io/projected/f193a6dd-401a-4763-bc0c-3ee20a960046-kube-api-access-pr7hb\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:37:37.795885 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:37.795799 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:37:37.998896 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:37.998860 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:37:38.098729 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:38.098639 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:37:38.203774 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:38.203728 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:37:38.295256 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:38.295214 2560 generic.go:356] "Generic (PLEG): container finished" podID="f193a6dd-401a-4763-bc0c-3ee20a960046" containerID="5a4cbae318d9a5e9cb6dd68f05628239a493aba4244e045c642d14e7ea5cda40" exitCode=0 Sep 10 13:37:38.295463 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:38.295264 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-3c32b1-86f4465b45-s8jd6" event={"ID":"f193a6dd-401a-4763-bc0c-3ee20a960046","Type":"ContainerDied","Data":"5a4cbae318d9a5e9cb6dd68f05628239a493aba4244e045c642d14e7ea5cda40"} Sep 10 13:37:38.295463 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:38.295301 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-3c32b1-86f4465b45-s8jd6" event={"ID":"f193a6dd-401a-4763-bc0c-3ee20a960046","Type":"ContainerDied","Data":"34153574e849d03f9c5e619633b009a9bdb1a7555a250c0a2528ec59a18afce4"} Sep 10 13:37:38.295463 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:38.295323 2560 scope.go:117] "RemoveContainer" containerID="5a4cbae318d9a5e9cb6dd68f05628239a493aba4244e045c642d14e7ea5cda40" Sep 10 13:37:38.295463 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:38.295267 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-3c32b1-86f4465b45-s8jd6" Sep 10 13:37:38.303498 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:38.303468 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:37:38.305343 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:38.305327 2560 scope.go:117] "RemoveContainer" containerID="5a4cbae318d9a5e9cb6dd68f05628239a493aba4244e045c642d14e7ea5cda40" Sep 10 13:37:38.305643 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:37:38.305623 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a4cbae318d9a5e9cb6dd68f05628239a493aba4244e045c642d14e7ea5cda40\": container with ID starting with 5a4cbae318d9a5e9cb6dd68f05628239a493aba4244e045c642d14e7ea5cda40 not found: ID does not exist" containerID="5a4cbae318d9a5e9cb6dd68f05628239a493aba4244e045c642d14e7ea5cda40" Sep 10 13:37:38.305689 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:38.305654 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a4cbae318d9a5e9cb6dd68f05628239a493aba4244e045c642d14e7ea5cda40"} err="failed to get container status \"5a4cbae318d9a5e9cb6dd68f05628239a493aba4244e045c642d14e7ea5cda40\": rpc error: code = NotFound desc = could not find container \"5a4cbae318d9a5e9cb6dd68f05628239a493aba4244e045c642d14e7ea5cda40\": container with ID starting with 5a4cbae318d9a5e9cb6dd68f05628239a493aba4244e045c642d14e7ea5cda40 not found: ID does not exist" Sep 10 13:37:38.323050 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:38.322990 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-3c32b1-86f4465b45-s8jd6"] Sep 10 13:37:38.325431 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:38.325395 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-3c32b1-86f4465b45-s8jd6"] Sep 10 13:37:38.562489 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:38.562458 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f193a6dd-401a-4763-bc0c-3ee20a960046" path="/var/lib/kubelet/pods/f193a6dd-401a-4763-bc0c-3ee20a960046/volumes" Sep 10 13:37:38.674172 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:38.674141 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:37:44.445970 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:44.445934 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:37:44.563949 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:44.563906 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:37:44.725679 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:44.725586 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:37:45.150970 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.150862 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:37:45.447806 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.447769 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr"] Sep 10 13:37:45.448364 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.448342 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f04e35a-f79e-43cc-91e7-64fd99c5a17c" containerName="istio-proxy" Sep 10 13:37:45.448468 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.448366 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f04e35a-f79e-43cc-91e7-64fd99c5a17c" containerName="istio-proxy" Sep 10 13:37:45.448468 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.448377 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97b7f072-3c7d-4241-838e-8baef91ae711" containerName="payload-processing" Sep 10 13:37:45.448468 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.448385 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="97b7f072-3c7d-4241-838e-8baef91ae711" containerName="payload-processing" Sep 10 13:37:45.448468 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.448422 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3301c0ac-5c0e-47ea-8ba3-d4d2e0065c93" containerName="payload-pre-processing" Sep 10 13:37:45.448468 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.448432 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3301c0ac-5c0e-47ea-8ba3-d4d2e0065c93" containerName="payload-pre-processing" Sep 10 13:37:45.448468 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.448461 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f04e35a-f79e-43cc-91e7-64fd99c5a17c" containerName="istio-proxy" Sep 10 13:37:45.448468 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.448469 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f04e35a-f79e-43cc-91e7-64fd99c5a17c" containerName="istio-proxy" Sep 10 13:37:45.448815 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.448489 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f193a6dd-401a-4763-bc0c-3ee20a960046" containerName="payload-processing" Sep 10 13:37:45.448815 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.448497 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f193a6dd-401a-4763-bc0c-3ee20a960046" containerName="payload-processing" Sep 10 13:37:45.448815 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.448603 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f193a6dd-401a-4763-bc0c-3ee20a960046" containerName="payload-processing" Sep 10 13:37:45.448815 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.448618 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="97b7f072-3c7d-4241-838e-8baef91ae711" containerName="payload-processing" Sep 10 13:37:45.448815 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.448629 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f04e35a-f79e-43cc-91e7-64fd99c5a17c" containerName="istio-proxy" Sep 10 13:37:45.448815 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.448639 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f04e35a-f79e-43cc-91e7-64fd99c5a17c" containerName="istio-proxy" Sep 10 13:37:45.448815 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.448650 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3301c0ac-5c0e-47ea-8ba3-d4d2e0065c93" containerName="payload-pre-processing" Sep 10 13:37:45.451998 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.451978 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" Sep 10 13:37:45.458141 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.458119 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-198b30-openshift-default-dockercfg-k9tv7\"" Sep 10 13:37:45.471251 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.471219 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr"] Sep 10 13:37:45.480871 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.480843 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:37:45.525285 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.525246 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq8g5\" (UniqueName: \"kubernetes.io/projected/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-kube-api-access-xq8g5\") pod \"e2e-noten-198b30-openshift-default-65cb9548b-k78dr\" (UID: \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\") " pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" Sep 10 13:37:45.525285 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.525284 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-credential-socket\") pod \"e2e-noten-198b30-openshift-default-65cb9548b-k78dr\" (UID: \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\") " pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" Sep 10 13:37:45.525532 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.525363 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-workload-certs\") pod \"e2e-noten-198b30-openshift-default-65cb9548b-k78dr\" (UID: \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\") " pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" Sep 10 13:37:45.525532 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.525391 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-istio-data\") pod \"e2e-noten-198b30-openshift-default-65cb9548b-k78dr\" (UID: \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\") " pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" Sep 10 13:37:45.525532 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.525446 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-workload-socket\") pod \"e2e-noten-198b30-openshift-default-65cb9548b-k78dr\" (UID: \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\") " pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" Sep 10 13:37:45.525532 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.525502 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-istio-podinfo\") pod \"e2e-noten-198b30-openshift-default-65cb9548b-k78dr\" (UID: \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\") " pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" Sep 10 13:37:45.525532 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.525532 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-istio-token\") pod \"e2e-noten-198b30-openshift-default-65cb9548b-k78dr\" (UID: \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\") " pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" Sep 10 13:37:45.525698 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.525558 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-istiod-ca-cert\") pod \"e2e-noten-198b30-openshift-default-65cb9548b-k78dr\" (UID: \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\") " pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" Sep 10 13:37:45.525698 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.525611 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-istio-envoy\") pod \"e2e-noten-198b30-openshift-default-65cb9548b-k78dr\" (UID: \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\") " pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" Sep 10 13:37:45.573669 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.573625 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:37:45.626920 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.626880 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-istio-data\") pod \"e2e-noten-198b30-openshift-default-65cb9548b-k78dr\" (UID: \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\") " pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" Sep 10 13:37:45.626920 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.626923 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-workload-socket\") pod \"e2e-noten-198b30-openshift-default-65cb9548b-k78dr\" (UID: \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\") " pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" Sep 10 13:37:45.627164 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.626974 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-istio-podinfo\") pod \"e2e-noten-198b30-openshift-default-65cb9548b-k78dr\" (UID: \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\") " pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" Sep 10 13:37:45.627164 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.626995 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-istio-token\") pod \"e2e-noten-198b30-openshift-default-65cb9548b-k78dr\" (UID: \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\") " pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" Sep 10 13:37:45.627164 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.627021 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-istiod-ca-cert\") pod \"e2e-noten-198b30-openshift-default-65cb9548b-k78dr\" (UID: \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\") " pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" Sep 10 13:37:45.627164 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.627041 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-istio-envoy\") pod \"e2e-noten-198b30-openshift-default-65cb9548b-k78dr\" (UID: \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\") " pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" Sep 10 13:37:45.627365 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.627203 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xq8g5\" (UniqueName: \"kubernetes.io/projected/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-kube-api-access-xq8g5\") pod \"e2e-noten-198b30-openshift-default-65cb9548b-k78dr\" (UID: \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\") " pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" Sep 10 13:37:45.627365 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.627228 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-credential-socket\") pod \"e2e-noten-198b30-openshift-default-65cb9548b-k78dr\" (UID: \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\") " pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" Sep 10 13:37:45.627365 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.627295 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-workload-certs\") pod \"e2e-noten-198b30-openshift-default-65cb9548b-k78dr\" (UID: \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\") " pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" Sep 10 13:37:45.627556 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.627544 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-workload-socket\") pod \"e2e-noten-198b30-openshift-default-65cb9548b-k78dr\" (UID: \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\") " pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" Sep 10 13:37:45.627616 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.627576 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-istio-data\") pod \"e2e-noten-198b30-openshift-default-65cb9548b-k78dr\" (UID: \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\") " pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" Sep 10 13:37:45.627780 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.627754 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-workload-certs\") pod \"e2e-noten-198b30-openshift-default-65cb9548b-k78dr\" (UID: \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\") " pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" Sep 10 13:37:45.627927 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.627905 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-credential-socket\") pod \"e2e-noten-198b30-openshift-default-65cb9548b-k78dr\" (UID: \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\") " pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" Sep 10 13:37:45.628167 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.628148 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-istiod-ca-cert\") pod \"e2e-noten-198b30-openshift-default-65cb9548b-k78dr\" (UID: \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\") " pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" Sep 10 13:37:45.629413 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.629377 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-istio-envoy\") pod \"e2e-noten-198b30-openshift-default-65cb9548b-k78dr\" (UID: \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\") " pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" Sep 10 13:37:45.629823 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.629803 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-istio-podinfo\") pod \"e2e-noten-198b30-openshift-default-65cb9548b-k78dr\" (UID: \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\") " pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" Sep 10 13:37:45.637235 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.637209 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq8g5\" (UniqueName: \"kubernetes.io/projected/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-kube-api-access-xq8g5\") pod \"e2e-noten-198b30-openshift-default-65cb9548b-k78dr\" (UID: \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\") " pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" Sep 10 13:37:45.637509 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.637489 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-istio-token\") pod \"e2e-noten-198b30-openshift-default-65cb9548b-k78dr\" (UID: \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\") " pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" Sep 10 13:37:45.763066 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:45.762975 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" Sep 10 13:37:46.051298 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:46.049005 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr"] Sep 10 13:37:46.052967 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:37:46.052939 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2b29abb_5e2f_4f9d_879d_5fd25a788ddf.slice/crio-4261ce010d260cc5863adc5b6a7d05952aadd363ae69fb1fc65eb7d79dfaf00b WatchSource:0}: Error finding container 4261ce010d260cc5863adc5b6a7d05952aadd363ae69fb1fc65eb7d79dfaf00b: Status 404 returned error can't find the container with id 4261ce010d260cc5863adc5b6a7d05952aadd363ae69fb1fc65eb7d79dfaf00b Sep 10 13:37:46.055611 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:46.055569 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:37:46.055734 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:46.055657 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:37:46.055734 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:46.055701 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:37:46.345456 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:46.345334 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" event={"ID":"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf","Type":"ContainerStarted","Data":"2c2295f5e91b1c208a6ac4c60eeee41b35a0bdec7e086f09f56eb6d52201c2e5"} Sep 10 13:37:46.345456 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:46.345418 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" event={"ID":"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf","Type":"ContainerStarted","Data":"4261ce010d260cc5863adc5b6a7d05952aadd363ae69fb1fc65eb7d79dfaf00b"} Sep 10 13:37:46.375196 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:46.375139 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" podStartSLOduration=1.375125109 podStartE2EDuration="1.375125109s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:37:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:37:46.370536688 +0000 UTC m=+1176.265254105" watchObservedRunningTime="2026-09-10 13:37:46.375125109 +0000 UTC m=+1176.269842528" Sep 10 13:37:46.734306 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:46.734272 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:37:46.763744 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:46.763712 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" Sep 10 13:37:46.884183 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:46.884141 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:37:47.046919 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:47.046829 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:37:47.769710 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:47.769679 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" Sep 10 13:37:48.357289 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:48.357254 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" Sep 10 13:37:48.358222 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:37:48.358202 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" Sep 10 13:38:02.824384 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:02.824348 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:38:04.679706 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:04.679662 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:38:10.556354 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:10.556319 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gtqlh_52fceedd-00f2-4943-ae47-dec51216d8e4/console-operator/2.log" Sep 10 13:38:10.559846 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:10.559818 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gtqlh_52fceedd-00f2-4943-ae47-dec51216d8e4/console-operator/2.log" Sep 10 13:38:11.666324 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:11.666289 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:38:11.774437 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:11.774379 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:38:11.889903 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:11.889872 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:38:18.963676 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:18.963640 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:38:19.032553 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:19.032514 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr"] Sep 10 13:38:19.032782 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:19.032757 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" podUID="f2b29abb-5e2f-4f9d-879d-5fd25a788ddf" containerName="istio-proxy" containerID="cri-o://2c2295f5e91b1c208a6ac4c60eeee41b35a0bdec7e086f09f56eb6d52201c2e5" gracePeriod=30 Sep 10 13:38:19.062632 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:19.062597 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:38:19.271009 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:19.270914 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:38:19.405984 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:19.405954 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:38:24.301733 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.301665 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" Sep 10 13:38:24.404713 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.404674 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-istio-data\") pod \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\" (UID: \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\") " Sep 10 13:38:24.404713 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.404718 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-istiod-ca-cert\") pod \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\" (UID: \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\") " Sep 10 13:38:24.404954 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.404746 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-istio-token\") pod \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\" (UID: \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\") " Sep 10 13:38:24.404954 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.404770 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-istio-envoy\") pod \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\" (UID: \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\") " Sep 10 13:38:24.404954 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.404813 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-workload-certs\") pod \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\" (UID: \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\") " Sep 10 13:38:24.404954 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.404890 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-istio-podinfo\") pod \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\" (UID: \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\") " Sep 10 13:38:24.404954 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.404922 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-workload-socket\") pod \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\" (UID: \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\") " Sep 10 13:38:24.405226 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.404960 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xq8g5\" (UniqueName: \"kubernetes.io/projected/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-kube-api-access-xq8g5\") pod \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\" (UID: \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\") " Sep 10 13:38:24.405226 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.404983 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-credential-socket\") pod \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\" (UID: \"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf\") " Sep 10 13:38:24.405226 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.405021 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-istio-data" (OuterVolumeSpecName: "istio-data") pod "f2b29abb-5e2f-4f9d-879d-5fd25a788ddf" (UID: "f2b29abb-5e2f-4f9d-879d-5fd25a788ddf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:38:24.405394 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.405176 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f2b29abb-5e2f-4f9d-879d-5fd25a788ddf" (UID: "f2b29abb-5e2f-4f9d-879d-5fd25a788ddf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:38:24.405394 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.405285 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f2b29abb-5e2f-4f9d-879d-5fd25a788ddf" (UID: "f2b29abb-5e2f-4f9d-879d-5fd25a788ddf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:38:24.405394 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.405182 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f2b29abb-5e2f-4f9d-879d-5fd25a788ddf" (UID: "f2b29abb-5e2f-4f9d-879d-5fd25a788ddf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:38:24.405605 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.405466 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-credential-socket\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:38:24.405605 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.405486 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-istio-data\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:38:24.405605 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.405500 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-istiod-ca-cert\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:38:24.405605 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.405513 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-workload-certs\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:38:24.405605 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.405568 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f2b29abb-5e2f-4f9d-879d-5fd25a788ddf" (UID: "f2b29abb-5e2f-4f9d-879d-5fd25a788ddf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:38:24.407162 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.407126 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f2b29abb-5e2f-4f9d-879d-5fd25a788ddf" (UID: "f2b29abb-5e2f-4f9d-879d-5fd25a788ddf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:38:24.407293 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.407169 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f2b29abb-5e2f-4f9d-879d-5fd25a788ddf" (UID: "f2b29abb-5e2f-4f9d-879d-5fd25a788ddf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 13:38:24.407457 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.407441 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-istio-token" (OuterVolumeSpecName: "istio-token") pod "f2b29abb-5e2f-4f9d-879d-5fd25a788ddf" (UID: "f2b29abb-5e2f-4f9d-879d-5fd25a788ddf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:38:24.407526 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.407475 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-kube-api-access-xq8g5" (OuterVolumeSpecName: "kube-api-access-xq8g5") pod "f2b29abb-5e2f-4f9d-879d-5fd25a788ddf" (UID: "f2b29abb-5e2f-4f9d-879d-5fd25a788ddf"). InnerVolumeSpecName "kube-api-access-xq8g5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:38:24.447490 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.447455 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:38:24.506603 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.506569 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-istio-podinfo\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:38:24.506603 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.506603 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-workload-socket\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:38:24.506777 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.506618 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xq8g5\" (UniqueName: \"kubernetes.io/projected/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-kube-api-access-xq8g5\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:38:24.506777 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.506628 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-istio-token\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:38:24.506777 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.506637 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf-istio-envoy\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:38:24.573223 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.573191 2560 generic.go:356] "Generic (PLEG): container finished" podID="f2b29abb-5e2f-4f9d-879d-5fd25a788ddf" containerID="2c2295f5e91b1c208a6ac4c60eeee41b35a0bdec7e086f09f56eb6d52201c2e5" exitCode=0 Sep 10 13:38:24.573347 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.573231 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" Sep 10 13:38:24.573347 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.573278 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" event={"ID":"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf","Type":"ContainerDied","Data":"2c2295f5e91b1c208a6ac4c60eeee41b35a0bdec7e086f09f56eb6d52201c2e5"} Sep 10 13:38:24.573347 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.573329 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr" event={"ID":"f2b29abb-5e2f-4f9d-879d-5fd25a788ddf","Type":"ContainerDied","Data":"4261ce010d260cc5863adc5b6a7d05952aadd363ae69fb1fc65eb7d79dfaf00b"} Sep 10 13:38:24.573488 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.573357 2560 scope.go:117] "RemoveContainer" containerID="2c2295f5e91b1c208a6ac4c60eeee41b35a0bdec7e086f09f56eb6d52201c2e5" Sep 10 13:38:24.583156 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.583131 2560 scope.go:117] "RemoveContainer" containerID="2c2295f5e91b1c208a6ac4c60eeee41b35a0bdec7e086f09f56eb6d52201c2e5" Sep 10 13:38:24.583469 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:38:24.583445 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c2295f5e91b1c208a6ac4c60eeee41b35a0bdec7e086f09f56eb6d52201c2e5\": container with ID starting with 2c2295f5e91b1c208a6ac4c60eeee41b35a0bdec7e086f09f56eb6d52201c2e5 not found: ID does not exist" containerID="2c2295f5e91b1c208a6ac4c60eeee41b35a0bdec7e086f09f56eb6d52201c2e5" Sep 10 13:38:24.583584 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.583481 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c2295f5e91b1c208a6ac4c60eeee41b35a0bdec7e086f09f56eb6d52201c2e5"} err="failed to get container status \"2c2295f5e91b1c208a6ac4c60eeee41b35a0bdec7e086f09f56eb6d52201c2e5\": rpc error: code = NotFound desc = could not find container \"2c2295f5e91b1c208a6ac4c60eeee41b35a0bdec7e086f09f56eb6d52201c2e5\": container with ID starting with 2c2295f5e91b1c208a6ac4c60eeee41b35a0bdec7e086f09f56eb6d52201c2e5 not found: ID does not exist" Sep 10 13:38:24.595923 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.595893 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr"] Sep 10 13:38:24.601575 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:24.601551 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-198b30-openshift-default-65cb9548b-k78dr"] Sep 10 13:38:26.563798 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:26.563766 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2b29abb-5e2f-4f9d-879d-5fd25a788ddf" path="/var/lib/kubelet/pods/f2b29abb-5e2f-4f9d-879d-5fd25a788ddf/volumes" Sep 10 13:38:27.777163 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:27.777125 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:38:35.101695 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:35.101652 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:38:35.279469 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:35.279433 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:38:35.434565 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:35.434528 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:38:35.718095 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:35.718000 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:38:36.795198 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:36.795163 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:38:36.880847 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:36.880811 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bc03be-g2gml"] Sep 10 13:38:36.881250 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:36.881237 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2b29abb-5e2f-4f9d-879d-5fd25a788ddf" containerName="istio-proxy" Sep 10 13:38:36.881297 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:36.881252 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2b29abb-5e2f-4f9d-879d-5fd25a788ddf" containerName="istio-proxy" Sep 10 13:38:36.881333 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:36.881316 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2b29abb-5e2f-4f9d-879d-5fd25a788ddf" containerName="istio-proxy" Sep 10 13:38:36.885985 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:36.885961 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bc03be-g2gml" Sep 10 13:38:36.889078 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:36.889052 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pffwn\"" Sep 10 13:38:36.889203 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:36.889058 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 13:38:36.897657 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:36.897626 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bc03be-g2gml"] Sep 10 13:38:37.010880 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:37.010851 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:38:37.031120 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:37.031079 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65219cc8-4d64-4fda-8708-5421ef1f3239-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-bc03be-g2gml\" (UID: \"65219cc8-4d64-4fda-8708-5421ef1f3239\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bc03be-g2gml" Sep 10 13:38:37.132059 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:37.131964 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65219cc8-4d64-4fda-8708-5421ef1f3239-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-bc03be-g2gml\" (UID: \"65219cc8-4d64-4fda-8708-5421ef1f3239\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bc03be-g2gml" Sep 10 13:38:37.132642 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:37.132624 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65219cc8-4d64-4fda-8708-5421ef1f3239-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-bc03be-g2gml\" (UID: \"65219cc8-4d64-4fda-8708-5421ef1f3239\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bc03be-g2gml" Sep 10 13:38:37.197387 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:37.197344 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bc03be-g2gml" Sep 10 13:38:37.482506 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:37.482468 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bc03be-g2gml"] Sep 10 13:38:37.485726 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:38:37.485691 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65219cc8_4d64_4fda_8708_5421ef1f3239.slice/crio-050af6938ce1b5815a522947ba6dc75a931d0394f4df9d873b01c598695442bc WatchSource:0}: Error finding container 050af6938ce1b5815a522947ba6dc75a931d0394f4df9d873b01c598695442bc: Status 404 returned error can't find the container with id 050af6938ce1b5815a522947ba6dc75a931d0394f4df9d873b01c598695442bc Sep 10 13:38:37.648925 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:37.648893 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bc03be-g2gml" event={"ID":"65219cc8-4d64-4fda-8708-5421ef1f3239","Type":"ContainerStarted","Data":"050af6938ce1b5815a522947ba6dc75a931d0394f4df9d873b01c598695442bc"} Sep 10 13:38:38.656498 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:38.656455 2560 generic.go:356] "Generic (PLEG): container finished" podID="65219cc8-4d64-4fda-8708-5421ef1f3239" containerID="0bf7f4c45b0af8efa1d0c06f46c447396113392fe69a8a601d4f15c4cdaad1ee" exitCode=0 Sep 10 13:38:38.656880 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:38.656523 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bc03be-g2gml" event={"ID":"65219cc8-4d64-4fda-8708-5421ef1f3239","Type":"ContainerDied","Data":"0bf7f4c45b0af8efa1d0c06f46c447396113392fe69a8a601d4f15c4cdaad1ee"} Sep 10 13:38:39.797569 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:39.797544 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bc03be-g2gml" Sep 10 13:38:39.857454 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:39.857418 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65219cc8-4d64-4fda-8708-5421ef1f3239-maas-api-service-ca\") pod \"65219cc8-4d64-4fda-8708-5421ef1f3239\" (UID: \"65219cc8-4d64-4fda-8708-5421ef1f3239\") " Sep 10 13:38:39.857829 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:39.857801 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65219cc8-4d64-4fda-8708-5421ef1f3239-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "65219cc8-4d64-4fda-8708-5421ef1f3239" (UID: "65219cc8-4d64-4fda-8708-5421ef1f3239"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:38:39.958178 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:39.958140 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65219cc8-4d64-4fda-8708-5421ef1f3239-maas-api-service-ca\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:38:40.669520 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:40.669476 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bc03be-g2gml" event={"ID":"65219cc8-4d64-4fda-8708-5421ef1f3239","Type":"ContainerDied","Data":"050af6938ce1b5815a522947ba6dc75a931d0394f4df9d873b01c598695442bc"} Sep 10 13:38:40.669520 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:40.669502 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bc03be-g2gml" Sep 10 13:38:40.669762 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:40.669549 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="050af6938ce1b5815a522947ba6dc75a931d0394f4df9d873b01c598695442bc" Sep 10 13:38:45.837207 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:45.837175 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:38:46.686091 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:46.686053 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:38:47.613297 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:47.613260 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:38:47.623735 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:47.623707 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-bc03be-58d4c9b479-rqw4t"] Sep 10 13:38:47.623987 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:47.623951 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-bc03be-58d4c9b479-rqw4t" podUID="27278e4d-9529-4f44-965e-e0a41074f5e3" containerName="payload-processing" containerID="cri-o://3a696f9498ba651149dc841b0cf05ea7c2ab970f0042e2e4019bd84bf10432d7" gracePeriod=30 Sep 10 13:38:47.636629 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:47.636598 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-bc03be-795f8655f4-w88qp"] Sep 10 13:38:47.636855 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:47.636829 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bc03be-795f8655f4-w88qp" podUID="9149a82b-029e-447f-a5ee-af1930e141a1" containerName="payload-pre-processing" containerID="cri-o://5be47c7dcd9799a96b7562c027916f562fa487966decbaf925f3444642e596bc" gracePeriod=30 Sep 10 13:38:47.711911 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:47.711872 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:38:47.818756 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:47.818513 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:38:47.842128 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:47.842069 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bc03be-g2gml"] Sep 10 13:38:47.846124 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:47.846093 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bc03be-g2gml"] Sep 10 13:38:47.899115 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:47.899088 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-bc03be-58d4c9b479-rqw4t" Sep 10 13:38:47.921710 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:47.921688 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bc03be-795f8655f4-w88qp" Sep 10 13:38:47.927092 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:47.927039 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:38:47.929306 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:47.929279 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sxtqd\" (UniqueName: \"kubernetes.io/projected/27278e4d-9529-4f44-965e-e0a41074f5e3-kube-api-access-sxtqd\") pod \"27278e4d-9529-4f44-965e-e0a41074f5e3\" (UID: \"27278e4d-9529-4f44-965e-e0a41074f5e3\") " Sep 10 13:38:47.929491 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:47.929380 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27278e4d-9529-4f44-965e-e0a41074f5e3-plugins-config-volume\") pod \"27278e4d-9529-4f44-965e-e0a41074f5e3\" (UID: \"27278e4d-9529-4f44-965e-e0a41074f5e3\") " Sep 10 13:38:47.929865 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:47.929838 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27278e4d-9529-4f44-965e-e0a41074f5e3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "27278e4d-9529-4f44-965e-e0a41074f5e3" (UID: "27278e4d-9529-4f44-965e-e0a41074f5e3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:38:47.931611 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:47.931583 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27278e4d-9529-4f44-965e-e0a41074f5e3-kube-api-access-sxtqd" (OuterVolumeSpecName: "kube-api-access-sxtqd") pod "27278e4d-9529-4f44-965e-e0a41074f5e3" (UID: "27278e4d-9529-4f44-965e-e0a41074f5e3"). InnerVolumeSpecName "kube-api-access-sxtqd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:38:48.030045 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:48.030006 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-szjmd\" (UniqueName: \"kubernetes.io/projected/9149a82b-029e-447f-a5ee-af1930e141a1-kube-api-access-szjmd\") pod \"9149a82b-029e-447f-a5ee-af1930e141a1\" (UID: \"9149a82b-029e-447f-a5ee-af1930e141a1\") " Sep 10 13:38:48.030218 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:48.030095 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9149a82b-029e-447f-a5ee-af1930e141a1-plugins-config-volume\") pod \"9149a82b-029e-447f-a5ee-af1930e141a1\" (UID: \"9149a82b-029e-447f-a5ee-af1930e141a1\") " Sep 10 13:38:48.030310 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:48.030298 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27278e4d-9529-4f44-965e-e0a41074f5e3-plugins-config-volume\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:38:48.030348 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:48.030311 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxtqd\" (UniqueName: \"kubernetes.io/projected/27278e4d-9529-4f44-965e-e0a41074f5e3-kube-api-access-sxtqd\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:38:48.030535 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:48.030511 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9149a82b-029e-447f-a5ee-af1930e141a1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9149a82b-029e-447f-a5ee-af1930e141a1" (UID: "9149a82b-029e-447f-a5ee-af1930e141a1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:38:48.032180 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:48.032160 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9149a82b-029e-447f-a5ee-af1930e141a1-kube-api-access-szjmd" (OuterVolumeSpecName: "kube-api-access-szjmd") pod "9149a82b-029e-447f-a5ee-af1930e141a1" (UID: "9149a82b-029e-447f-a5ee-af1930e141a1"). InnerVolumeSpecName "kube-api-access-szjmd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:38:48.131517 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:48.131435 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9149a82b-029e-447f-a5ee-af1930e141a1-plugins-config-volume\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:38:48.131517 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:48.131465 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szjmd\" (UniqueName: \"kubernetes.io/projected/9149a82b-029e-447f-a5ee-af1930e141a1-kube-api-access-szjmd\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:38:48.266696 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:48.266662 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:38:48.563190 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:48.563153 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65219cc8-4d64-4fda-8708-5421ef1f3239" path="/var/lib/kubelet/pods/65219cc8-4d64-4fda-8708-5421ef1f3239/volumes" Sep 10 13:38:48.719981 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:48.719942 2560 generic.go:356] "Generic (PLEG): container finished" podID="9149a82b-029e-447f-a5ee-af1930e141a1" containerID="5be47c7dcd9799a96b7562c027916f562fa487966decbaf925f3444642e596bc" exitCode=0 Sep 10 13:38:48.719981 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:48.719985 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bc03be-795f8655f4-w88qp" Sep 10 13:38:48.720503 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:48.720027 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bc03be-795f8655f4-w88qp" event={"ID":"9149a82b-029e-447f-a5ee-af1930e141a1","Type":"ContainerDied","Data":"5be47c7dcd9799a96b7562c027916f562fa487966decbaf925f3444642e596bc"} Sep 10 13:38:48.720503 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:48.720077 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bc03be-795f8655f4-w88qp" event={"ID":"9149a82b-029e-447f-a5ee-af1930e141a1","Type":"ContainerDied","Data":"87aab48a4917f9bc6f1792248e0d017f50487cd4986b40bac98bd842e1aa28b4"} Sep 10 13:38:48.720503 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:48.720104 2560 scope.go:117] "RemoveContainer" containerID="5be47c7dcd9799a96b7562c027916f562fa487966decbaf925f3444642e596bc" Sep 10 13:38:48.721623 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:48.721598 2560 generic.go:356] "Generic (PLEG): container finished" podID="27278e4d-9529-4f44-965e-e0a41074f5e3" containerID="3a696f9498ba651149dc841b0cf05ea7c2ab970f0042e2e4019bd84bf10432d7" exitCode=0 Sep 10 13:38:48.721689 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:48.721641 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-bc03be-58d4c9b479-rqw4t" Sep 10 13:38:48.721744 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:48.721686 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-bc03be-58d4c9b479-rqw4t" event={"ID":"27278e4d-9529-4f44-965e-e0a41074f5e3","Type":"ContainerDied","Data":"3a696f9498ba651149dc841b0cf05ea7c2ab970f0042e2e4019bd84bf10432d7"} Sep 10 13:38:48.721798 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:48.721750 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-bc03be-58d4c9b479-rqw4t" event={"ID":"27278e4d-9529-4f44-965e-e0a41074f5e3","Type":"ContainerDied","Data":"54ad281b47f05a4d176435090a25b615934b0c9d4f5e7ee4bfaf7d9d0ba54fcc"} Sep 10 13:38:48.729309 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:48.729293 2560 scope.go:117] "RemoveContainer" containerID="5be47c7dcd9799a96b7562c027916f562fa487966decbaf925f3444642e596bc" Sep 10 13:38:48.729684 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:38:48.729656 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5be47c7dcd9799a96b7562c027916f562fa487966decbaf925f3444642e596bc\": container with ID starting with 5be47c7dcd9799a96b7562c027916f562fa487966decbaf925f3444642e596bc not found: ID does not exist" containerID="5be47c7dcd9799a96b7562c027916f562fa487966decbaf925f3444642e596bc" Sep 10 13:38:48.729787 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:48.729688 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5be47c7dcd9799a96b7562c027916f562fa487966decbaf925f3444642e596bc"} err="failed to get container status \"5be47c7dcd9799a96b7562c027916f562fa487966decbaf925f3444642e596bc\": rpc error: code = NotFound desc = could not find container \"5be47c7dcd9799a96b7562c027916f562fa487966decbaf925f3444642e596bc\": container with ID starting with 5be47c7dcd9799a96b7562c027916f562fa487966decbaf925f3444642e596bc not found: ID does not exist" Sep 10 13:38:48.729787 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:48.729706 2560 scope.go:117] "RemoveContainer" containerID="3a696f9498ba651149dc841b0cf05ea7c2ab970f0042e2e4019bd84bf10432d7" Sep 10 13:38:48.737712 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:48.737692 2560 scope.go:117] "RemoveContainer" containerID="3a696f9498ba651149dc841b0cf05ea7c2ab970f0042e2e4019bd84bf10432d7" Sep 10 13:38:48.738006 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:38:48.737988 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a696f9498ba651149dc841b0cf05ea7c2ab970f0042e2e4019bd84bf10432d7\": container with ID starting with 3a696f9498ba651149dc841b0cf05ea7c2ab970f0042e2e4019bd84bf10432d7 not found: ID does not exist" containerID="3a696f9498ba651149dc841b0cf05ea7c2ab970f0042e2e4019bd84bf10432d7" Sep 10 13:38:48.738056 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:48.738013 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a696f9498ba651149dc841b0cf05ea7c2ab970f0042e2e4019bd84bf10432d7"} err="failed to get container status \"3a696f9498ba651149dc841b0cf05ea7c2ab970f0042e2e4019bd84bf10432d7\": rpc error: code = NotFound desc = could not find container \"3a696f9498ba651149dc841b0cf05ea7c2ab970f0042e2e4019bd84bf10432d7\": container with ID starting with 3a696f9498ba651149dc841b0cf05ea7c2ab970f0042e2e4019bd84bf10432d7 not found: ID does not exist" Sep 10 13:38:48.741881 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:48.741857 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-bc03be-795f8655f4-w88qp"] Sep 10 13:38:48.746782 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:48.746760 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-bc03be-795f8655f4-w88qp"] Sep 10 13:38:48.759000 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:48.758972 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-bc03be-58d4c9b479-rqw4t"] Sep 10 13:38:48.766397 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:48.766373 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-bc03be-58d4c9b479-rqw4t"] Sep 10 13:38:50.570102 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:50.570066 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27278e4d-9529-4f44-965e-e0a41074f5e3" path="/var/lib/kubelet/pods/27278e4d-9529-4f44-965e-e0a41074f5e3/volumes" Sep 10 13:38:50.570612 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:50.570550 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9149a82b-029e-447f-a5ee-af1930e141a1" path="/var/lib/kubelet/pods/9149a82b-029e-447f-a5ee-af1930e141a1/volumes" Sep 10 13:38:54.230011 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:54.229976 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:38:54.466860 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:54.466825 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:38:54.726129 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:54.726089 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:38:55.298700 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:55.298661 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:38:55.443608 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:55.443575 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:38:57.036311 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:57.036281 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:38:57.135893 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:57.135862 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:38:57.235248 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:57.235203 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:38:57.399996 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:57.399912 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp"] Sep 10 13:38:57.400144 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:57.400123 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" podUID="ef2fd3f9-1e16-4c56-95da-af9379f5ec52" containerName="istio-proxy" containerID="cri-o://516f8205c971edd6329ff4a096def270dfe62bbe53cf8ddca8c82a7192a49e8c" gracePeriod=30 Sep 10 13:38:57.417538 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:57.417509 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:38:57.725766 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:57.725724 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d6159c-mfq7n"] Sep 10 13:38:57.726798 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:57.726764 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65219cc8-4d64-4fda-8708-5421ef1f3239" containerName="revoke-keys" Sep 10 13:38:57.726798 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:57.726797 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="65219cc8-4d64-4fda-8708-5421ef1f3239" containerName="revoke-keys" Sep 10 13:38:57.726953 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:57.726812 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9149a82b-029e-447f-a5ee-af1930e141a1" containerName="payload-pre-processing" Sep 10 13:38:57.726953 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:57.726821 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9149a82b-029e-447f-a5ee-af1930e141a1" containerName="payload-pre-processing" Sep 10 13:38:57.726953 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:57.726847 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27278e4d-9529-4f44-965e-e0a41074f5e3" containerName="payload-processing" Sep 10 13:38:57.726953 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:57.726857 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="27278e4d-9529-4f44-965e-e0a41074f5e3" containerName="payload-processing" Sep 10 13:38:57.727153 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:57.727024 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="27278e4d-9529-4f44-965e-e0a41074f5e3" containerName="payload-processing" Sep 10 13:38:57.727153 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:57.727048 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="65219cc8-4d64-4fda-8708-5421ef1f3239" containerName="revoke-keys" Sep 10 13:38:57.727153 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:57.727061 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9149a82b-029e-447f-a5ee-af1930e141a1" containerName="payload-pre-processing" Sep 10 13:38:57.732778 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:57.732751 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d6159c-mfq7n" Sep 10 13:38:57.735709 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:57.735686 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 13:38:57.735933 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:57.735917 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pffwn\"" Sep 10 13:38:57.742032 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:57.742010 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d6159c-mfq7n"] Sep 10 13:38:57.819193 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:57.819154 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9eb8fc54-fb98-4173-aeb7-7ee48442d5a8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-d6159c-mfq7n\" (UID: \"9eb8fc54-fb98-4173-aeb7-7ee48442d5a8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d6159c-mfq7n" Sep 10 13:38:57.920753 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:57.920719 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9eb8fc54-fb98-4173-aeb7-7ee48442d5a8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-d6159c-mfq7n\" (UID: \"9eb8fc54-fb98-4173-aeb7-7ee48442d5a8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d6159c-mfq7n" Sep 10 13:38:57.921312 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:57.921292 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9eb8fc54-fb98-4173-aeb7-7ee48442d5a8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-d6159c-mfq7n\" (UID: \"9eb8fc54-fb98-4173-aeb7-7ee48442d5a8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d6159c-mfq7n" Sep 10 13:38:58.045443 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:58.045332 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d6159c-mfq7n" Sep 10 13:38:58.089895 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:58.089858 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:38:58.297886 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:58.297793 2560 patch_prober.go:28] interesting pod/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 10 13:38:58.297886 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:58.297874 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" podUID="ef2fd3f9-1e16-4c56-95da-af9379f5ec52" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 13:38:58.333800 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:58.333773 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d6159c-mfq7n"] Sep 10 13:38:58.335809 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:38:58.335781 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eb8fc54_fb98_4173_aeb7_7ee48442d5a8.slice/crio-5db21b6cddb524fe50955cfa3e78399ff789dc4be52927ae3da2bdffa637e8fd WatchSource:0}: Error finding container 5db21b6cddb524fe50955cfa3e78399ff789dc4be52927ae3da2bdffa637e8fd: Status 404 returned error can't find the container with id 5db21b6cddb524fe50955cfa3e78399ff789dc4be52927ae3da2bdffa637e8fd Sep 10 13:38:58.783374 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:58.783337 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d6159c-mfq7n" event={"ID":"9eb8fc54-fb98-4173-aeb7-7ee48442d5a8","Type":"ContainerStarted","Data":"630a6db4ef3639ae6483258a99d1f643861c7b51532a005ca552a168c5493deb"} Sep 10 13:38:58.783572 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:58.783416 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d6159c-mfq7n" event={"ID":"9eb8fc54-fb98-4173-aeb7-7ee48442d5a8","Type":"ContainerStarted","Data":"5db21b6cddb524fe50955cfa3e78399ff789dc4be52927ae3da2bdffa637e8fd"} Sep 10 13:38:58.814049 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:58.813933 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d6159c-mfq7n" podStartSLOduration=1.813916399 podStartE2EDuration="1.813916399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:38:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:38:58.812887117 +0000 UTC m=+1248.707604535" watchObservedRunningTime="2026-09-10 13:38:58.813916399 +0000 UTC m=+1248.708633817" Sep 10 13:38:59.790484 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:59.790450 2560 generic.go:356] "Generic (PLEG): container finished" podID="9eb8fc54-fb98-4173-aeb7-7ee48442d5a8" containerID="630a6db4ef3639ae6483258a99d1f643861c7b51532a005ca552a168c5493deb" exitCode=0 Sep 10 13:38:59.790866 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:38:59.790538 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d6159c-mfq7n" event={"ID":"9eb8fc54-fb98-4173-aeb7-7ee48442d5a8","Type":"ContainerDied","Data":"630a6db4ef3639ae6483258a99d1f643861c7b51532a005ca552a168c5493deb"} Sep 10 13:39:00.936862 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:00.936833 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d6159c-mfq7n" Sep 10 13:39:01.048204 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:01.048112 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9eb8fc54-fb98-4173-aeb7-7ee48442d5a8-maas-api-service-ca\") pod \"9eb8fc54-fb98-4173-aeb7-7ee48442d5a8\" (UID: \"9eb8fc54-fb98-4173-aeb7-7ee48442d5a8\") " Sep 10 13:39:01.048513 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:01.048490 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9eb8fc54-fb98-4173-aeb7-7ee48442d5a8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9eb8fc54-fb98-4173-aeb7-7ee48442d5a8" (UID: "9eb8fc54-fb98-4173-aeb7-7ee48442d5a8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:39:01.149777 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:01.149736 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9eb8fc54-fb98-4173-aeb7-7ee48442d5a8-maas-api-service-ca\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:39:01.804092 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:01.804062 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d6159c-mfq7n" Sep 10 13:39:01.804092 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:01.804072 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d6159c-mfq7n" event={"ID":"9eb8fc54-fb98-4173-aeb7-7ee48442d5a8","Type":"ContainerDied","Data":"5db21b6cddb524fe50955cfa3e78399ff789dc4be52927ae3da2bdffa637e8fd"} Sep 10 13:39:01.804311 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:01.804118 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5db21b6cddb524fe50955cfa3e78399ff789dc4be52927ae3da2bdffa637e8fd" Sep 10 13:39:02.664107 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.664049 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:39:02.765551 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.765508 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-istio-token\") pod \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\" (UID: \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\") " Sep 10 13:39:02.765740 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.765566 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-workload-certs\") pod \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\" (UID: \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\") " Sep 10 13:39:02.765740 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.765602 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-istiod-ca-cert\") pod \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\" (UID: \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\") " Sep 10 13:39:02.765740 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.765717 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-workload-socket\") pod \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\" (UID: \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\") " Sep 10 13:39:02.765902 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.765773 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-istio-podinfo\") pod \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\" (UID: \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\") " Sep 10 13:39:02.765902 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.765792 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-istio-data\") pod \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\" (UID: \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\") " Sep 10 13:39:02.765902 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.765840 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-credential-socket\") pod \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\" (UID: \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\") " Sep 10 13:39:02.765902 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.765899 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-istio-envoy\") pod \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\" (UID: \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\") " Sep 10 13:39:02.766120 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.765939 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76l6h\" (UniqueName: \"kubernetes.io/projected/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-kube-api-access-76l6h\") pod \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\" (UID: \"ef2fd3f9-1e16-4c56-95da-af9379f5ec52\") " Sep 10 13:39:02.766120 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.765977 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ef2fd3f9-1e16-4c56-95da-af9379f5ec52" (UID: "ef2fd3f9-1e16-4c56-95da-af9379f5ec52"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:39:02.766120 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.766050 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ef2fd3f9-1e16-4c56-95da-af9379f5ec52" (UID: "ef2fd3f9-1e16-4c56-95da-af9379f5ec52"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:39:02.766120 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.765984 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ef2fd3f9-1e16-4c56-95da-af9379f5ec52" (UID: "ef2fd3f9-1e16-4c56-95da-af9379f5ec52"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:39:02.766338 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.766118 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-istio-data" (OuterVolumeSpecName: "istio-data") pod "ef2fd3f9-1e16-4c56-95da-af9379f5ec52" (UID: "ef2fd3f9-1e16-4c56-95da-af9379f5ec52"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:39:02.766338 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.766328 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-workload-certs\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:39:02.766485 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.766377 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-istiod-ca-cert\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:39:02.766485 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.766392 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-workload-socket\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:39:02.766485 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.766432 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-istio-data\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:39:02.766485 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.766440 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ef2fd3f9-1e16-4c56-95da-af9379f5ec52" (UID: "ef2fd3f9-1e16-4c56-95da-af9379f5ec52"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:39:02.767961 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.767935 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ef2fd3f9-1e16-4c56-95da-af9379f5ec52" (UID: "ef2fd3f9-1e16-4c56-95da-af9379f5ec52"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 13:39:02.768073 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.768050 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-kube-api-access-76l6h" (OuterVolumeSpecName: "kube-api-access-76l6h") pod "ef2fd3f9-1e16-4c56-95da-af9379f5ec52" (UID: "ef2fd3f9-1e16-4c56-95da-af9379f5ec52"). InnerVolumeSpecName "kube-api-access-76l6h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:39:02.768197 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.768172 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-istio-token" (OuterVolumeSpecName: "istio-token") pod "ef2fd3f9-1e16-4c56-95da-af9379f5ec52" (UID: "ef2fd3f9-1e16-4c56-95da-af9379f5ec52"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:39:02.768482 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.768455 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ef2fd3f9-1e16-4c56-95da-af9379f5ec52" (UID: "ef2fd3f9-1e16-4c56-95da-af9379f5ec52"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:39:02.811961 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.811921 2560 generic.go:356] "Generic (PLEG): container finished" podID="ef2fd3f9-1e16-4c56-95da-af9379f5ec52" containerID="516f8205c971edd6329ff4a096def270dfe62bbe53cf8ddca8c82a7192a49e8c" exitCode=0 Sep 10 13:39:02.812110 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.811984 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" Sep 10 13:39:02.812110 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.812008 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" event={"ID":"ef2fd3f9-1e16-4c56-95da-af9379f5ec52","Type":"ContainerDied","Data":"516f8205c971edd6329ff4a096def270dfe62bbe53cf8ddca8c82a7192a49e8c"} Sep 10 13:39:02.812110 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.812099 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp" event={"ID":"ef2fd3f9-1e16-4c56-95da-af9379f5ec52","Type":"ContainerDied","Data":"503f0d4fb045cc1976f553e543c7dbbb72c45c7a89e422b8aa0336cdc30a3ac4"} Sep 10 13:39:02.812241 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.812129 2560 scope.go:117] "RemoveContainer" containerID="516f8205c971edd6329ff4a096def270dfe62bbe53cf8ddca8c82a7192a49e8c" Sep 10 13:39:02.822627 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.822609 2560 scope.go:117] "RemoveContainer" containerID="3d1df39b4bf4f7bb9d2e3a0a57871be6c046aff9432557f63e4d17ee9b09cada" Sep 10 13:39:02.831469 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.831447 2560 scope.go:117] "RemoveContainer" containerID="516f8205c971edd6329ff4a096def270dfe62bbe53cf8ddca8c82a7192a49e8c" Sep 10 13:39:02.831820 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:39:02.831799 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"516f8205c971edd6329ff4a096def270dfe62bbe53cf8ddca8c82a7192a49e8c\": container with ID starting with 516f8205c971edd6329ff4a096def270dfe62bbe53cf8ddca8c82a7192a49e8c not found: ID does not exist" containerID="516f8205c971edd6329ff4a096def270dfe62bbe53cf8ddca8c82a7192a49e8c" Sep 10 13:39:02.831889 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.831830 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"516f8205c971edd6329ff4a096def270dfe62bbe53cf8ddca8c82a7192a49e8c"} err="failed to get container status \"516f8205c971edd6329ff4a096def270dfe62bbe53cf8ddca8c82a7192a49e8c\": rpc error: code = NotFound desc = could not find container \"516f8205c971edd6329ff4a096def270dfe62bbe53cf8ddca8c82a7192a49e8c\": container with ID starting with 516f8205c971edd6329ff4a096def270dfe62bbe53cf8ddca8c82a7192a49e8c not found: ID does not exist" Sep 10 13:39:02.831889 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.831850 2560 scope.go:117] "RemoveContainer" containerID="3d1df39b4bf4f7bb9d2e3a0a57871be6c046aff9432557f63e4d17ee9b09cada" Sep 10 13:39:02.832107 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:39:02.832094 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d1df39b4bf4f7bb9d2e3a0a57871be6c046aff9432557f63e4d17ee9b09cada\": container with ID starting with 3d1df39b4bf4f7bb9d2e3a0a57871be6c046aff9432557f63e4d17ee9b09cada not found: ID does not exist" containerID="3d1df39b4bf4f7bb9d2e3a0a57871be6c046aff9432557f63e4d17ee9b09cada" Sep 10 13:39:02.832149 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.832110 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d1df39b4bf4f7bb9d2e3a0a57871be6c046aff9432557f63e4d17ee9b09cada"} err="failed to get container status \"3d1df39b4bf4f7bb9d2e3a0a57871be6c046aff9432557f63e4d17ee9b09cada\": rpc error: code = NotFound desc = could not find container \"3d1df39b4bf4f7bb9d2e3a0a57871be6c046aff9432557f63e4d17ee9b09cada\": container with ID starting with 3d1df39b4bf4f7bb9d2e3a0a57871be6c046aff9432557f63e4d17ee9b09cada not found: ID does not exist" Sep 10 13:39:02.837337 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.837309 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp"] Sep 10 13:39:02.844950 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.844911 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-bc03be-openshift-default-65cd54fbcb-xkmcp"] Sep 10 13:39:02.867849 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.867819 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-istio-token\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:39:02.867849 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.867849 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-istio-podinfo\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:39:02.868018 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.867861 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-credential-socket\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:39:02.868018 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.867875 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-istio-envoy\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:39:02.868018 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:02.867888 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76l6h\" (UniqueName: \"kubernetes.io/projected/ef2fd3f9-1e16-4c56-95da-af9379f5ec52-kube-api-access-76l6h\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:39:03.813729 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:03.813691 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:04.563632 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:04.563591 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef2fd3f9-1e16-4c56-95da-af9379f5ec52" path="/var/lib/kubelet/pods/ef2fd3f9-1e16-4c56-95da-af9379f5ec52/volumes" Sep 10 13:39:05.176785 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:05.176748 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:05.341126 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:05.341092 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:05.937786 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:05.935289 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:09.032032 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:09.031996 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:09.089949 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:09.089911 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d6159c-564486d59d-scfzx"] Sep 10 13:39:09.090316 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:09.090284 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d6159c-564486d59d-scfzx" podUID="ef9ad2ff-b9fd-4db0-84b6-fa25678dfb3a" containerName="payload-pre-processing" containerID="cri-o://e489c5a259442a8b2259d5dc987a2af015d41d320f7220004f443048b8a0f6be" gracePeriod=30 Sep 10 13:39:09.187824 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:09.187789 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:09.355856 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:09.355814 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d6159c-564486d59d-scfzx" Sep 10 13:39:09.388351 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:09.388322 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:09.403971 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:09.403940 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d6159c-mfq7n"] Sep 10 13:39:09.407838 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:09.407811 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-d6159c-mfq7n"] Sep 10 13:39:09.425964 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:09.425937 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef9ad2ff-b9fd-4db0-84b6-fa25678dfb3a-plugins-config-volume\") pod \"ef9ad2ff-b9fd-4db0-84b6-fa25678dfb3a\" (UID: \"ef9ad2ff-b9fd-4db0-84b6-fa25678dfb3a\") " Sep 10 13:39:09.426074 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:09.426006 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dp4g2\" (UniqueName: \"kubernetes.io/projected/ef9ad2ff-b9fd-4db0-84b6-fa25678dfb3a-kube-api-access-dp4g2\") pod \"ef9ad2ff-b9fd-4db0-84b6-fa25678dfb3a\" (UID: \"ef9ad2ff-b9fd-4db0-84b6-fa25678dfb3a\") " Sep 10 13:39:09.426312 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:09.426289 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef9ad2ff-b9fd-4db0-84b6-fa25678dfb3a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ef9ad2ff-b9fd-4db0-84b6-fa25678dfb3a" (UID: "ef9ad2ff-b9fd-4db0-84b6-fa25678dfb3a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:39:09.428465 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:09.428435 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef9ad2ff-b9fd-4db0-84b6-fa25678dfb3a-kube-api-access-dp4g2" (OuterVolumeSpecName: "kube-api-access-dp4g2") pod "ef9ad2ff-b9fd-4db0-84b6-fa25678dfb3a" (UID: "ef9ad2ff-b9fd-4db0-84b6-fa25678dfb3a"). InnerVolumeSpecName "kube-api-access-dp4g2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:39:09.527113 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:09.527072 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ef9ad2ff-b9fd-4db0-84b6-fa25678dfb3a-plugins-config-volume\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:39:09.527113 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:09.527104 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dp4g2\" (UniqueName: \"kubernetes.io/projected/ef9ad2ff-b9fd-4db0-84b6-fa25678dfb3a-kube-api-access-dp4g2\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:39:09.580202 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:09.580169 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:09.805252 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:09.805222 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:09.853842 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:09.853808 2560 generic.go:356] "Generic (PLEG): container finished" podID="ef9ad2ff-b9fd-4db0-84b6-fa25678dfb3a" containerID="e489c5a259442a8b2259d5dc987a2af015d41d320f7220004f443048b8a0f6be" exitCode=0 Sep 10 13:39:09.854038 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:09.853856 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d6159c-564486d59d-scfzx" Sep 10 13:39:09.854038 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:09.853905 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d6159c-564486d59d-scfzx" event={"ID":"ef9ad2ff-b9fd-4db0-84b6-fa25678dfb3a","Type":"ContainerDied","Data":"e489c5a259442a8b2259d5dc987a2af015d41d320f7220004f443048b8a0f6be"} Sep 10 13:39:09.854038 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:09.853958 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d6159c-564486d59d-scfzx" event={"ID":"ef9ad2ff-b9fd-4db0-84b6-fa25678dfb3a","Type":"ContainerDied","Data":"90e2bdf4d85f046d95f0f29f241924fc1727f8b8f77880ec0732e323b4834d0d"} Sep 10 13:39:09.854038 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:09.853985 2560 scope.go:117] "RemoveContainer" containerID="e489c5a259442a8b2259d5dc987a2af015d41d320f7220004f443048b8a0f6be" Sep 10 13:39:09.863858 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:09.863840 2560 scope.go:117] "RemoveContainer" containerID="e489c5a259442a8b2259d5dc987a2af015d41d320f7220004f443048b8a0f6be" Sep 10 13:39:09.864204 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:39:09.864184 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e489c5a259442a8b2259d5dc987a2af015d41d320f7220004f443048b8a0f6be\": container with ID starting with e489c5a259442a8b2259d5dc987a2af015d41d320f7220004f443048b8a0f6be not found: ID does not exist" containerID="e489c5a259442a8b2259d5dc987a2af015d41d320f7220004f443048b8a0f6be" Sep 10 13:39:09.864281 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:09.864217 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e489c5a259442a8b2259d5dc987a2af015d41d320f7220004f443048b8a0f6be"} err="failed to get container status \"e489c5a259442a8b2259d5dc987a2af015d41d320f7220004f443048b8a0f6be\": rpc error: code = NotFound desc = could not find container \"e489c5a259442a8b2259d5dc987a2af015d41d320f7220004f443048b8a0f6be\": container with ID starting with e489c5a259442a8b2259d5dc987a2af015d41d320f7220004f443048b8a0f6be not found: ID does not exist" Sep 10 13:39:09.879717 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:09.879683 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d6159c-564486d59d-scfzx"] Sep 10 13:39:09.884844 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:09.884813 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d6159c-564486d59d-scfzx"] Sep 10 13:39:10.563864 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:10.563830 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9eb8fc54-fb98-4173-aeb7-7ee48442d5a8" path="/var/lib/kubelet/pods/9eb8fc54-fb98-4173-aeb7-7ee48442d5a8/volumes" Sep 10 13:39:10.564208 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:10.564180 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef9ad2ff-b9fd-4db0-84b6-fa25678dfb3a" path="/var/lib/kubelet/pods/ef9ad2ff-b9fd-4db0-84b6-fa25678dfb3a/volumes" Sep 10 13:39:13.858026 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:13.857991 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:13.963014 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:13.962979 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:14.159444 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.159317 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:14.458376 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.458344 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w"] Sep 10 13:39:14.458775 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.458760 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef9ad2ff-b9fd-4db0-84b6-fa25678dfb3a" containerName="payload-pre-processing" Sep 10 13:39:14.458843 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.458776 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef9ad2ff-b9fd-4db0-84b6-fa25678dfb3a" containerName="payload-pre-processing" Sep 10 13:39:14.458843 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.458800 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef2fd3f9-1e16-4c56-95da-af9379f5ec52" containerName="istio-proxy" Sep 10 13:39:14.458843 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.458809 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef2fd3f9-1e16-4c56-95da-af9379f5ec52" containerName="istio-proxy" Sep 10 13:39:14.458843 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.458832 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9eb8fc54-fb98-4173-aeb7-7ee48442d5a8" containerName="revoke-keys" Sep 10 13:39:14.458843 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.458838 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9eb8fc54-fb98-4173-aeb7-7ee48442d5a8" containerName="revoke-keys" Sep 10 13:39:14.459027 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.458900 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef2fd3f9-1e16-4c56-95da-af9379f5ec52" containerName="istio-proxy" Sep 10 13:39:14.459027 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.458908 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9eb8fc54-fb98-4173-aeb7-7ee48442d5a8" containerName="revoke-keys" Sep 10 13:39:14.459027 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.458915 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef2fd3f9-1e16-4c56-95da-af9379f5ec52" containerName="istio-proxy" Sep 10 13:39:14.459027 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.458922 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef9ad2ff-b9fd-4db0-84b6-fa25678dfb3a" containerName="payload-pre-processing" Sep 10 13:39:14.463620 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.463598 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:39:14.468946 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.468911 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-839f01ff-openshift-default-dockercfg-bjwkf\"" Sep 10 13:39:14.501610 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.501574 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w"] Sep 10 13:39:14.527467 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.527435 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:14.574059 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.574022 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b697811-e443-42cf-bc5e-09a9919417cc-istio-token\") pod \"e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w\" (UID: \"6b697811-e443-42cf-bc5e-09a9919417cc\") " pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:39:14.574225 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.574087 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b697811-e443-42cf-bc5e-09a9919417cc-istiod-ca-cert\") pod \"e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w\" (UID: \"6b697811-e443-42cf-bc5e-09a9919417cc\") " pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:39:14.574225 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.574112 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b697811-e443-42cf-bc5e-09a9919417cc-istio-podinfo\") pod \"e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w\" (UID: \"6b697811-e443-42cf-bc5e-09a9919417cc\") " pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:39:14.574225 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.574185 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b697811-e443-42cf-bc5e-09a9919417cc-workload-certs\") pod \"e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w\" (UID: \"6b697811-e443-42cf-bc5e-09a9919417cc\") " pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:39:14.574385 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.574253 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frkqc\" (UniqueName: \"kubernetes.io/projected/6b697811-e443-42cf-bc5e-09a9919417cc-kube-api-access-frkqc\") pod \"e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w\" (UID: \"6b697811-e443-42cf-bc5e-09a9919417cc\") " pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:39:14.574385 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.574291 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b697811-e443-42cf-bc5e-09a9919417cc-istio-envoy\") pod \"e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w\" (UID: \"6b697811-e443-42cf-bc5e-09a9919417cc\") " pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:39:14.574385 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.574326 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b697811-e443-42cf-bc5e-09a9919417cc-credential-socket\") pod \"e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w\" (UID: \"6b697811-e443-42cf-bc5e-09a9919417cc\") " pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:39:14.574385 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.574347 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b697811-e443-42cf-bc5e-09a9919417cc-workload-socket\") pod \"e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w\" (UID: \"6b697811-e443-42cf-bc5e-09a9919417cc\") " pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:39:14.574549 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.574394 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b697811-e443-42cf-bc5e-09a9919417cc-istio-data\") pod \"e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w\" (UID: \"6b697811-e443-42cf-bc5e-09a9919417cc\") " pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:39:14.675654 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.675613 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b697811-e443-42cf-bc5e-09a9919417cc-istio-envoy\") pod \"e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w\" (UID: \"6b697811-e443-42cf-bc5e-09a9919417cc\") " pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:39:14.675818 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.675677 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b697811-e443-42cf-bc5e-09a9919417cc-credential-socket\") pod \"e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w\" (UID: \"6b697811-e443-42cf-bc5e-09a9919417cc\") " pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:39:14.675818 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.675700 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b697811-e443-42cf-bc5e-09a9919417cc-workload-socket\") pod \"e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w\" (UID: \"6b697811-e443-42cf-bc5e-09a9919417cc\") " pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:39:14.675818 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.675733 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b697811-e443-42cf-bc5e-09a9919417cc-istio-data\") pod \"e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w\" (UID: \"6b697811-e443-42cf-bc5e-09a9919417cc\") " pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:39:14.676004 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.675896 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b697811-e443-42cf-bc5e-09a9919417cc-istio-token\") pod \"e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w\" (UID: \"6b697811-e443-42cf-bc5e-09a9919417cc\") " pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:39:14.676004 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.675966 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b697811-e443-42cf-bc5e-09a9919417cc-istiod-ca-cert\") pod \"e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w\" (UID: \"6b697811-e443-42cf-bc5e-09a9919417cc\") " pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:39:14.676107 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.676004 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b697811-e443-42cf-bc5e-09a9919417cc-istio-podinfo\") pod \"e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w\" (UID: \"6b697811-e443-42cf-bc5e-09a9919417cc\") " pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:39:14.676107 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.676044 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b697811-e443-42cf-bc5e-09a9919417cc-workload-certs\") pod \"e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w\" (UID: \"6b697811-e443-42cf-bc5e-09a9919417cc\") " pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:39:14.676107 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.676077 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b697811-e443-42cf-bc5e-09a9919417cc-credential-socket\") pod \"e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w\" (UID: \"6b697811-e443-42cf-bc5e-09a9919417cc\") " pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:39:14.676244 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.676084 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frkqc\" (UniqueName: \"kubernetes.io/projected/6b697811-e443-42cf-bc5e-09a9919417cc-kube-api-access-frkqc\") pod \"e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w\" (UID: \"6b697811-e443-42cf-bc5e-09a9919417cc\") " pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:39:14.676244 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.676119 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b697811-e443-42cf-bc5e-09a9919417cc-workload-socket\") pod \"e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w\" (UID: \"6b697811-e443-42cf-bc5e-09a9919417cc\") " pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:39:14.676244 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.676227 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b697811-e443-42cf-bc5e-09a9919417cc-istio-data\") pod \"e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w\" (UID: \"6b697811-e443-42cf-bc5e-09a9919417cc\") " pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:39:14.676528 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.676503 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b697811-e443-42cf-bc5e-09a9919417cc-workload-certs\") pod \"e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w\" (UID: \"6b697811-e443-42cf-bc5e-09a9919417cc\") " pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:39:14.676754 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.676733 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b697811-e443-42cf-bc5e-09a9919417cc-istiod-ca-cert\") pod \"e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w\" (UID: \"6b697811-e443-42cf-bc5e-09a9919417cc\") " pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:39:14.678152 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.678131 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b697811-e443-42cf-bc5e-09a9919417cc-istio-envoy\") pod \"e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w\" (UID: \"6b697811-e443-42cf-bc5e-09a9919417cc\") " pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:39:14.678987 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.678967 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b697811-e443-42cf-bc5e-09a9919417cc-istio-podinfo\") pod \"e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w\" (UID: \"6b697811-e443-42cf-bc5e-09a9919417cc\") " pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:39:14.708975 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.708897 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b697811-e443-42cf-bc5e-09a9919417cc-istio-token\") pod \"e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w\" (UID: \"6b697811-e443-42cf-bc5e-09a9919417cc\") " pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:39:14.709844 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.709824 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frkqc\" (UniqueName: \"kubernetes.io/projected/6b697811-e443-42cf-bc5e-09a9919417cc-kube-api-access-frkqc\") pod \"e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w\" (UID: \"6b697811-e443-42cf-bc5e-09a9919417cc\") " pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:39:14.774167 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.774137 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:39:14.895726 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:14.895690 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:15.067426 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:15.067383 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w"] Sep 10 13:39:15.068650 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:39:15.068624 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b697811_e443_42cf_bc5e_09a9919417cc.slice/crio-e2e761688ee6a250f1b1c216064a3fd0133aa4b761c96081027406b8f4d00449 WatchSource:0}: Error finding container e2e761688ee6a250f1b1c216064a3fd0133aa4b761c96081027406b8f4d00449: Status 404 returned error can't find the container with id e2e761688ee6a250f1b1c216064a3fd0133aa4b761c96081027406b8f4d00449 Sep 10 13:39:15.070702 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:15.070671 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:39:15.070773 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:15.070737 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:39:15.070773 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:15.070764 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:39:15.887908 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:15.887867 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" event={"ID":"6b697811-e443-42cf-bc5e-09a9919417cc","Type":"ContainerStarted","Data":"258f63a02b64c82799b7ac231f982cf1ec4c3e70785163a0e9307f2ec9d4f0e2"} Sep 10 13:39:15.888104 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:15.887928 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" event={"ID":"6b697811-e443-42cf-bc5e-09a9919417cc","Type":"ContainerStarted","Data":"e2e761688ee6a250f1b1c216064a3fd0133aa4b761c96081027406b8f4d00449"} Sep 10 13:39:15.927903 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:15.927849 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" podStartSLOduration=1.927832146 podStartE2EDuration="1.927832146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:39:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:39:15.925987424 +0000 UTC m=+1265.820704842" watchObservedRunningTime="2026-09-10 13:39:15.927832146 +0000 UTC m=+1265.822549564" Sep 10 13:39:16.774975 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:16.774929 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:39:16.779581 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:16.779553 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:39:16.893745 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:16.893708 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:39:16.894839 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:16.894816 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:39:17.924570 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:17.924537 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:18.141663 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:18.141627 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:18.243036 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:18.242945 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:18.349288 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:18.349248 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:18.562169 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:18.562071 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:19.100723 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.100689 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:19.225171 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.225141 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:19.326883 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.326840 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:19.486601 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.486567 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm"] Sep 10 13:39:19.486952 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.486940 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef2fd3f9-1e16-4c56-95da-af9379f5ec52" containerName="istio-proxy" Sep 10 13:39:19.486995 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.486954 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef2fd3f9-1e16-4c56-95da-af9379f5ec52" containerName="istio-proxy" Sep 10 13:39:19.490053 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.490032 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:39:19.493081 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.493057 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-767dce-openshift-default-dockercfg-lmz55\"" Sep 10 13:39:19.509073 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.509045 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:19.520804 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.520768 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm"] Sep 10 13:39:19.622065 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.622027 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38e4531c-cd84-4583-8cc8-e86e966c398e-istio-data\") pod \"e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm\" (UID: \"38e4531c-cd84-4583-8cc8-e86e966c398e\") " pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:39:19.622265 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.622074 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38e4531c-cd84-4583-8cc8-e86e966c398e-istio-podinfo\") pod \"e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm\" (UID: \"38e4531c-cd84-4583-8cc8-e86e966c398e\") " pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:39:19.622265 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.622102 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrwvw\" (UniqueName: \"kubernetes.io/projected/38e4531c-cd84-4583-8cc8-e86e966c398e-kube-api-access-jrwvw\") pod \"e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm\" (UID: \"38e4531c-cd84-4583-8cc8-e86e966c398e\") " pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:39:19.622265 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.622160 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38e4531c-cd84-4583-8cc8-e86e966c398e-istio-token\") pod \"e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm\" (UID: \"38e4531c-cd84-4583-8cc8-e86e966c398e\") " pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:39:19.622265 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.622225 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38e4531c-cd84-4583-8cc8-e86e966c398e-workload-socket\") pod \"e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm\" (UID: \"38e4531c-cd84-4583-8cc8-e86e966c398e\") " pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:39:19.622265 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.622255 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38e4531c-cd84-4583-8cc8-e86e966c398e-istiod-ca-cert\") pod \"e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm\" (UID: \"38e4531c-cd84-4583-8cc8-e86e966c398e\") " pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:39:19.622532 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.622297 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38e4531c-cd84-4583-8cc8-e86e966c398e-workload-certs\") pod \"e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm\" (UID: \"38e4531c-cd84-4583-8cc8-e86e966c398e\") " pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:39:19.622532 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.622323 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38e4531c-cd84-4583-8cc8-e86e966c398e-istio-envoy\") pod \"e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm\" (UID: \"38e4531c-cd84-4583-8cc8-e86e966c398e\") " pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:39:19.622532 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.622362 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38e4531c-cd84-4583-8cc8-e86e966c398e-credential-socket\") pod \"e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm\" (UID: \"38e4531c-cd84-4583-8cc8-e86e966c398e\") " pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:39:19.628175 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.628148 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:19.723967 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.723922 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38e4531c-cd84-4583-8cc8-e86e966c398e-workload-socket\") pod \"e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm\" (UID: \"38e4531c-cd84-4583-8cc8-e86e966c398e\") " pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:39:19.724157 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.723983 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38e4531c-cd84-4583-8cc8-e86e966c398e-istiod-ca-cert\") pod \"e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm\" (UID: \"38e4531c-cd84-4583-8cc8-e86e966c398e\") " pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:39:19.724157 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.724032 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38e4531c-cd84-4583-8cc8-e86e966c398e-workload-certs\") pod \"e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm\" (UID: \"38e4531c-cd84-4583-8cc8-e86e966c398e\") " pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:39:19.724157 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.724056 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38e4531c-cd84-4583-8cc8-e86e966c398e-istio-envoy\") pod \"e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm\" (UID: \"38e4531c-cd84-4583-8cc8-e86e966c398e\") " pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:39:19.724157 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.724096 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38e4531c-cd84-4583-8cc8-e86e966c398e-credential-socket\") pod \"e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm\" (UID: \"38e4531c-cd84-4583-8cc8-e86e966c398e\") " pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:39:19.724157 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.724151 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38e4531c-cd84-4583-8cc8-e86e966c398e-istio-data\") pod \"e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm\" (UID: \"38e4531c-cd84-4583-8cc8-e86e966c398e\") " pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:39:19.724478 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.724187 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38e4531c-cd84-4583-8cc8-e86e966c398e-istio-podinfo\") pod \"e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm\" (UID: \"38e4531c-cd84-4583-8cc8-e86e966c398e\") " pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:39:19.724478 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.724216 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrwvw\" (UniqueName: \"kubernetes.io/projected/38e4531c-cd84-4583-8cc8-e86e966c398e-kube-api-access-jrwvw\") pod \"e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm\" (UID: \"38e4531c-cd84-4583-8cc8-e86e966c398e\") " pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:39:19.724478 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.724263 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38e4531c-cd84-4583-8cc8-e86e966c398e-istio-token\") pod \"e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm\" (UID: \"38e4531c-cd84-4583-8cc8-e86e966c398e\") " pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:39:19.724478 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.724368 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38e4531c-cd84-4583-8cc8-e86e966c398e-workload-socket\") pod \"e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm\" (UID: \"38e4531c-cd84-4583-8cc8-e86e966c398e\") " pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:39:19.724700 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.724672 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38e4531c-cd84-4583-8cc8-e86e966c398e-credential-socket\") pod \"e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm\" (UID: \"38e4531c-cd84-4583-8cc8-e86e966c398e\") " pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:39:19.724847 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.724804 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38e4531c-cd84-4583-8cc8-e86e966c398e-istiod-ca-cert\") pod \"e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm\" (UID: \"38e4531c-cd84-4583-8cc8-e86e966c398e\") " pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:39:19.724957 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.724922 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38e4531c-cd84-4583-8cc8-e86e966c398e-istio-data\") pod \"e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm\" (UID: \"38e4531c-cd84-4583-8cc8-e86e966c398e\") " pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:39:19.725768 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.725741 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38e4531c-cd84-4583-8cc8-e86e966c398e-workload-certs\") pod \"e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm\" (UID: \"38e4531c-cd84-4583-8cc8-e86e966c398e\") " pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:39:19.727780 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.727728 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38e4531c-cd84-4583-8cc8-e86e966c398e-istio-podinfo\") pod \"e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm\" (UID: \"38e4531c-cd84-4583-8cc8-e86e966c398e\") " pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:39:19.727969 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.727901 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38e4531c-cd84-4583-8cc8-e86e966c398e-istio-envoy\") pod \"e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm\" (UID: \"38e4531c-cd84-4583-8cc8-e86e966c398e\") " pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:39:19.735563 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.735538 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38e4531c-cd84-4583-8cc8-e86e966c398e-istio-token\") pod \"e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm\" (UID: \"38e4531c-cd84-4583-8cc8-e86e966c398e\") " pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:39:19.735656 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.735573 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrwvw\" (UniqueName: \"kubernetes.io/projected/38e4531c-cd84-4583-8cc8-e86e966c398e-kube-api-access-jrwvw\") pod \"e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm\" (UID: \"38e4531c-cd84-4583-8cc8-e86e966c398e\") " pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:39:19.756879 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.756802 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:19.800206 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:19.800160 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:39:20.046845 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:20.046757 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:20.103556 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:20.103530 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm"] Sep 10 13:39:20.105005 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:39:20.104973 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38e4531c_cd84_4583_8cc8_e86e966c398e.slice/crio-fd2c68fab4bb0dd1ba3337019df908b10cf07efce43ae4cfe18947d5c3c73f1d WatchSource:0}: Error finding container fd2c68fab4bb0dd1ba3337019df908b10cf07efce43ae4cfe18947d5c3c73f1d: Status 404 returned error can't find the container with id fd2c68fab4bb0dd1ba3337019df908b10cf07efce43ae4cfe18947d5c3c73f1d Sep 10 13:39:20.107501 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:20.107460 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:39:20.107569 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:20.107549 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:39:20.107616 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:20.107595 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:39:20.198838 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:20.198803 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:20.917197 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:20.917155 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" event={"ID":"38e4531c-cd84-4583-8cc8-e86e966c398e","Type":"ContainerStarted","Data":"ec93289e8ef5174fba424e190e88d387b06e6efc0f9b221630238ab2e45fb357"} Sep 10 13:39:20.917476 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:20.917241 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" event={"ID":"38e4531c-cd84-4583-8cc8-e86e966c398e","Type":"ContainerStarted","Data":"fd2c68fab4bb0dd1ba3337019df908b10cf07efce43ae4cfe18947d5c3c73f1d"} Sep 10 13:39:20.963045 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:20.962989 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" podStartSLOduration=1.962969996 podStartE2EDuration="1.962969996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:39:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:39:20.960728857 +0000 UTC m=+1270.855446278" watchObservedRunningTime="2026-09-10 13:39:20.962969996 +0000 UTC m=+1270.857687413" Sep 10 13:39:21.800958 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:21.800911 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:39:21.805460 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:21.805433 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:39:21.922948 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:21.922916 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:39:21.923993 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:21.923968 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:39:23.101750 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:23.101708 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:23.240490 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:23.240451 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:24.218463 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:24.218427 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:24.320763 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:24.320728 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:24.979880 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:24.979841 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:25.093234 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:25.093188 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:25.203021 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:25.202985 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-767dce-56ffd89574-cptzg"] Sep 10 13:39:25.206920 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:25.206901 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-767dce-56ffd89574-cptzg" Sep 10 13:39:25.209711 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:25.209685 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-767dce-dockercfg-4zxsg\"" Sep 10 13:39:25.209868 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:25.209848 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-767dce\"" Sep 10 13:39:25.230608 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:25.230524 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-767dce-56ffd89574-cptzg"] Sep 10 13:39:25.282195 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:25.282012 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8374fc76-574f-48bd-aaf8-021f1e4e5b43-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-767dce-56ffd89574-cptzg\" (UID: \"8374fc76-574f-48bd-aaf8-021f1e4e5b43\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-767dce-56ffd89574-cptzg" Sep 10 13:39:25.282195 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:25.282111 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hh6v\" (UniqueName: \"kubernetes.io/projected/8374fc76-574f-48bd-aaf8-021f1e4e5b43-kube-api-access-2hh6v\") pod \"payload-pre-processing-e2e-shared-a-w3-767dce-56ffd89574-cptzg\" (UID: \"8374fc76-574f-48bd-aaf8-021f1e4e5b43\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-767dce-56ffd89574-cptzg" Sep 10 13:39:25.292286 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:25.292244 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-767dce-6db9bbd447-j75c5"] Sep 10 13:39:25.296079 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:25.296061 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-767dce-6db9bbd447-j75c5" Sep 10 13:39:25.311064 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:25.311032 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-767dce-6db9bbd447-j75c5"] Sep 10 13:39:25.383034 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:25.382995 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hh6v\" (UniqueName: \"kubernetes.io/projected/8374fc76-574f-48bd-aaf8-021f1e4e5b43-kube-api-access-2hh6v\") pod \"payload-pre-processing-e2e-shared-a-w3-767dce-56ffd89574-cptzg\" (UID: \"8374fc76-574f-48bd-aaf8-021f1e4e5b43\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-767dce-56ffd89574-cptzg" Sep 10 13:39:25.383228 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:25.383064 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb9cd903-e9a3-4af1-b443-da43b25180ab-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-767dce-6db9bbd447-j75c5\" (UID: \"fb9cd903-e9a3-4af1-b443-da43b25180ab\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-767dce-6db9bbd447-j75c5" Sep 10 13:39:25.383228 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:25.383092 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vxjl\" (UniqueName: \"kubernetes.io/projected/fb9cd903-e9a3-4af1-b443-da43b25180ab-kube-api-access-5vxjl\") pod \"payload-processing-e2e-shared-a-w3-767dce-6db9bbd447-j75c5\" (UID: \"fb9cd903-e9a3-4af1-b443-da43b25180ab\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-767dce-6db9bbd447-j75c5" Sep 10 13:39:25.383228 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:25.383206 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8374fc76-574f-48bd-aaf8-021f1e4e5b43-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-767dce-56ffd89574-cptzg\" (UID: \"8374fc76-574f-48bd-aaf8-021f1e4e5b43\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-767dce-56ffd89574-cptzg" Sep 10 13:39:25.383941 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:25.383922 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8374fc76-574f-48bd-aaf8-021f1e4e5b43-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-767dce-56ffd89574-cptzg\" (UID: \"8374fc76-574f-48bd-aaf8-021f1e4e5b43\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-767dce-56ffd89574-cptzg" Sep 10 13:39:25.397024 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:25.396993 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hh6v\" (UniqueName: \"kubernetes.io/projected/8374fc76-574f-48bd-aaf8-021f1e4e5b43-kube-api-access-2hh6v\") pod \"payload-pre-processing-e2e-shared-a-w3-767dce-56ffd89574-cptzg\" (UID: \"8374fc76-574f-48bd-aaf8-021f1e4e5b43\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-767dce-56ffd89574-cptzg" Sep 10 13:39:25.484295 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:25.484196 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb9cd903-e9a3-4af1-b443-da43b25180ab-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-767dce-6db9bbd447-j75c5\" (UID: \"fb9cd903-e9a3-4af1-b443-da43b25180ab\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-767dce-6db9bbd447-j75c5" Sep 10 13:39:25.484295 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:25.484237 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vxjl\" (UniqueName: \"kubernetes.io/projected/fb9cd903-e9a3-4af1-b443-da43b25180ab-kube-api-access-5vxjl\") pod \"payload-processing-e2e-shared-a-w3-767dce-6db9bbd447-j75c5\" (UID: \"fb9cd903-e9a3-4af1-b443-da43b25180ab\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-767dce-6db9bbd447-j75c5" Sep 10 13:39:25.484949 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:25.484926 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb9cd903-e9a3-4af1-b443-da43b25180ab-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-767dce-6db9bbd447-j75c5\" (UID: \"fb9cd903-e9a3-4af1-b443-da43b25180ab\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-767dce-6db9bbd447-j75c5" Sep 10 13:39:25.495293 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:25.495256 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vxjl\" (UniqueName: \"kubernetes.io/projected/fb9cd903-e9a3-4af1-b443-da43b25180ab-kube-api-access-5vxjl\") pod \"payload-processing-e2e-shared-a-w3-767dce-6db9bbd447-j75c5\" (UID: \"fb9cd903-e9a3-4af1-b443-da43b25180ab\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-767dce-6db9bbd447-j75c5" Sep 10 13:39:25.519086 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:25.519045 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-767dce-56ffd89574-cptzg" Sep 10 13:39:25.612027 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:25.611689 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-767dce-6db9bbd447-j75c5" Sep 10 13:39:25.827932 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:39:25.827391 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8374fc76_574f_48bd_aaf8_021f1e4e5b43.slice/crio-3ba2dbcb502163d379539248ce8443605bf2836037086168b3054078387218fb WatchSource:0}: Error finding container 3ba2dbcb502163d379539248ce8443605bf2836037086168b3054078387218fb: Status 404 returned error can't find the container with id 3ba2dbcb502163d379539248ce8443605bf2836037086168b3054078387218fb Sep 10 13:39:25.827932 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:25.827720 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-767dce-56ffd89574-cptzg"] Sep 10 13:39:25.922219 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:25.922019 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:25.924117 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:25.924088 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-767dce-6db9bbd447-j75c5"] Sep 10 13:39:25.925024 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:39:25.924993 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb9cd903_e9a3_4af1_b443_da43b25180ab.slice/crio-f9ff30b2f9ac4ceecc292095c9717be39152b04c83a9d84a020706cd0d403dca WatchSource:0}: Error finding container f9ff30b2f9ac4ceecc292095c9717be39152b04c83a9d84a020706cd0d403dca: Status 404 returned error can't find the container with id f9ff30b2f9ac4ceecc292095c9717be39152b04c83a9d84a020706cd0d403dca Sep 10 13:39:25.955996 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:25.953161 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-767dce-56ffd89574-cptzg" event={"ID":"8374fc76-574f-48bd-aaf8-021f1e4e5b43","Type":"ContainerStarted","Data":"fc5e2a2efa77f7ca7dc8daac5cbaaaf3e3f42967aaf4699d16eff6bb65284b8a"} Sep 10 13:39:25.955996 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:25.953245 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-767dce-56ffd89574-cptzg" event={"ID":"8374fc76-574f-48bd-aaf8-021f1e4e5b43","Type":"ContainerStarted","Data":"3ba2dbcb502163d379539248ce8443605bf2836037086168b3054078387218fb"} Sep 10 13:39:25.955996 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:25.954167 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-767dce-56ffd89574-cptzg" Sep 10 13:39:25.955996 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:25.955754 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-767dce-6db9bbd447-j75c5" event={"ID":"fb9cd903-e9a3-4af1-b443-da43b25180ab","Type":"ContainerStarted","Data":"f9ff30b2f9ac4ceecc292095c9717be39152b04c83a9d84a020706cd0d403dca"} Sep 10 13:39:25.985609 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:25.985552 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-767dce-56ffd89574-cptzg" podStartSLOduration=0.985532745 podStartE2EDuration="985.532745ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:39:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:39:25.984243835 +0000 UTC m=+1275.878961254" watchObservedRunningTime="2026-09-10 13:39:25.985532745 +0000 UTC m=+1275.880250164" Sep 10 13:39:26.967489 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:26.967446 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-767dce-6db9bbd447-j75c5" event={"ID":"fb9cd903-e9a3-4af1-b443-da43b25180ab","Type":"ContainerStarted","Data":"b472aec8a204239260be1909a3f7ba878777b5ce7f7e6f575725fb3c1d31e7ff"} Sep 10 13:39:26.968007 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:26.967539 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-767dce-6db9bbd447-j75c5" Sep 10 13:39:26.989914 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:26.989847 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-767dce-6db9bbd447-j75c5" podStartSLOduration=1.9898253590000001 podStartE2EDuration="1.989825359s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:39:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:39:26.986692762 +0000 UTC m=+1276.881410201" watchObservedRunningTime="2026-09-10 13:39:26.989825359 +0000 UTC m=+1276.884542781" Sep 10 13:39:31.895337 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:31.895240 2560 patch_prober.go:28] interesting pod/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Sep 10 13:39:31.895337 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:31.895301 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" podUID="6b697811-e443-42cf-bc5e-09a9919417cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Sep 10 13:39:31.998987 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:31.998950 2560 generic.go:356] "Generic (PLEG): container finished" podID="6b697811-e443-42cf-bc5e-09a9919417cc" containerID="258f63a02b64c82799b7ac231f982cf1ec4c3e70785163a0e9307f2ec9d4f0e2" exitCode=0 Sep 10 13:39:31.999166 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:31.999021 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" event={"ID":"6b697811-e443-42cf-bc5e-09a9919417cc","Type":"ContainerDied","Data":"258f63a02b64c82799b7ac231f982cf1ec4c3e70785163a0e9307f2ec9d4f0e2"} Sep 10 13:39:31.999541 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:31.999526 2560 scope.go:117] "RemoveContainer" containerID="258f63a02b64c82799b7ac231f982cf1ec4c3e70785163a0e9307f2ec9d4f0e2" Sep 10 13:39:32.000710 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:32.000680 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:39:32.000816 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:32.000750 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:39:32.000816 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:32.000799 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:39:33.007367 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:33.007324 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" event={"ID":"6b697811-e443-42cf-bc5e-09a9919417cc","Type":"ContainerStarted","Data":"e283bfe1118fa6cefc36af1edd9eb4281fcb8b4ae012b3ee792f642af46c9b65"} Sep 10 13:39:33.299307 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:33.299161 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:33.774568 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:33.774530 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:39:33.774783 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:33.774656 2560 patch_prober.go:28] interesting pod/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Sep 10 13:39:33.774783 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:33.774700 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" podUID="6b697811-e443-42cf-bc5e-09a9919417cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Sep 10 13:39:34.775321 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:34.775273 2560 patch_prober.go:28] interesting pod/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Sep 10 13:39:34.775860 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:34.775355 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" podUID="6b697811-e443-42cf-bc5e-09a9919417cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Sep 10 13:39:35.775219 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:35.775117 2560 patch_prober.go:28] interesting pod/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Sep 10 13:39:35.775219 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:35.775199 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" podUID="6b697811-e443-42cf-bc5e-09a9919417cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Sep 10 13:39:36.775499 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:36.775450 2560 patch_prober.go:28] interesting pod/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Sep 10 13:39:36.776044 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:36.775522 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" podUID="6b697811-e443-42cf-bc5e-09a9919417cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Sep 10 13:39:37.775329 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:37.775292 2560 patch_prober.go:28] interesting pod/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Sep 10 13:39:37.775524 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:37.775348 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" podUID="6b697811-e443-42cf-bc5e-09a9919417cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Sep 10 13:39:37.963616 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:37.963572 2560 patch_prober.go:28] interesting pod/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 10 13:39:37.963809 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:37.963644 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" podUID="38e4531c-cd84-4583-8cc8-e86e966c398e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 10 13:39:37.976421 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:37.976379 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-767dce-56ffd89574-cptzg" Sep 10 13:39:38.778706 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:38.778668 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:39:38.779202 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:38.779184 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:39:38.779903 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:38.779886 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:39:38.981604 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:38.981546 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-767dce-6db9bbd447-j75c5" Sep 10 13:39:40.057135 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:40.057097 2560 generic.go:356] "Generic (PLEG): container finished" podID="38e4531c-cd84-4583-8cc8-e86e966c398e" containerID="ec93289e8ef5174fba424e190e88d387b06e6efc0f9b221630238ab2e45fb357" exitCode=0 Sep 10 13:39:40.057569 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:40.057168 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" event={"ID":"38e4531c-cd84-4583-8cc8-e86e966c398e","Type":"ContainerDied","Data":"ec93289e8ef5174fba424e190e88d387b06e6efc0f9b221630238ab2e45fb357"} Sep 10 13:39:40.057669 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:40.057655 2560 scope.go:117] "RemoveContainer" containerID="ec93289e8ef5174fba424e190e88d387b06e6efc0f9b221630238ab2e45fb357" Sep 10 13:39:40.058647 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:40.058624 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:39:40.058711 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:40.058688 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:39:40.058748 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:40.058719 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:39:41.070831 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:41.070793 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" event={"ID":"38e4531c-cd84-4583-8cc8-e86e966c398e","Type":"ContainerStarted","Data":"d26665bb15f4853cd187feb1e0d13d5ea4788a59e2f42214d10f450b09e86a80"} Sep 10 13:39:41.801268 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:41.801235 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:39:41.801462 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:41.801436 2560 patch_prober.go:28] interesting pod/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Sep 10 13:39:41.801514 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:41.801499 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" podUID="38e4531c-cd84-4583-8cc8-e86e966c398e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Sep 10 13:39:42.358010 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:42.357964 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:42.800865 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:42.800826 2560 patch_prober.go:28] interesting pod/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Sep 10 13:39:42.801049 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:42.800883 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" podUID="38e4531c-cd84-4583-8cc8-e86e966c398e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Sep 10 13:39:43.801279 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:43.801237 2560 patch_prober.go:28] interesting pod/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Sep 10 13:39:43.801695 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:43.801311 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" podUID="38e4531c-cd84-4583-8cc8-e86e966c398e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Sep 10 13:39:44.800970 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:44.800934 2560 patch_prober.go:28] interesting pod/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Sep 10 13:39:44.801156 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:44.801004 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" podUID="38e4531c-cd84-4583-8cc8-e86e966c398e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Sep 10 13:39:45.642212 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:45.642166 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:45.800869 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:45.800830 2560 patch_prober.go:28] interesting pod/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Sep 10 13:39:45.801044 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:45.800891 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" podUID="38e4531c-cd84-4583-8cc8-e86e966c398e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Sep 10 13:39:46.804749 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:46.804718 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:39:46.805217 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:46.804996 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:39:46.805813 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:46.805797 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:39:48.048320 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:48.048262 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:48.149425 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:48.149343 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:48.424965 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:48.424935 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:48.718100 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:48.718003 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:48.964354 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:48.964315 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:49.098551 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:49.098473 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:49.214049 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:49.214020 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:52.222471 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:52.222435 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:52.326733 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:52.326699 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:53.717947 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:53.717903 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:53.929947 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:53.929905 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:54.039217 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:54.039097 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:54.511866 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:54.511833 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:54.610710 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:54.610671 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:54.697815 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:54.697782 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-89be92-6f87d675d6-jldxc"] Sep 10 13:39:54.700774 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:54.700751 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-89be92-6f87d675d6-jldxc" Sep 10 13:39:54.703551 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:54.703529 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-89be92\"" Sep 10 13:39:54.703640 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:54.703579 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-89be92-dockercfg-rldqg\"" Sep 10 13:39:54.713832 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:54.713804 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-89be92-6f87d675d6-jldxc"] Sep 10 13:39:54.793777 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:54.793699 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-89be92-98b7b59cf-9krl9"] Sep 10 13:39:54.796243 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:54.796188 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-89be92-98b7b59cf-9krl9" Sep 10 13:39:54.798066 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:54.798041 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgm9l\" (UniqueName: \"kubernetes.io/projected/be8293d4-b9ef-4d0f-9686-ae65c483f0f4-kube-api-access-bgm9l\") pod \"payload-pre-processing-e2e-shared-b-w3-89be92-6f87d675d6-jldxc\" (UID: \"be8293d4-b9ef-4d0f-9686-ae65c483f0f4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-89be92-6f87d675d6-jldxc" Sep 10 13:39:54.798154 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:54.798094 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/be8293d4-b9ef-4d0f-9686-ae65c483f0f4-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-89be92-6f87d675d6-jldxc\" (UID: \"be8293d4-b9ef-4d0f-9686-ae65c483f0f4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-89be92-6f87d675d6-jldxc" Sep 10 13:39:54.806650 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:54.806625 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-89be92-98b7b59cf-9krl9"] Sep 10 13:39:54.899478 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:54.899440 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/be8293d4-b9ef-4d0f-9686-ae65c483f0f4-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-89be92-6f87d675d6-jldxc\" (UID: \"be8293d4-b9ef-4d0f-9686-ae65c483f0f4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-89be92-6f87d675d6-jldxc" Sep 10 13:39:54.899667 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:54.899514 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d11b889b-57c3-4ae5-8cce-f337308aaa6d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-89be92-98b7b59cf-9krl9\" (UID: \"d11b889b-57c3-4ae5-8cce-f337308aaa6d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-89be92-98b7b59cf-9krl9" Sep 10 13:39:54.899667 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:54.899652 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tplhn\" (UniqueName: \"kubernetes.io/projected/d11b889b-57c3-4ae5-8cce-f337308aaa6d-kube-api-access-tplhn\") pod \"payload-processing-e2e-shared-b-w3-89be92-98b7b59cf-9krl9\" (UID: \"d11b889b-57c3-4ae5-8cce-f337308aaa6d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-89be92-98b7b59cf-9krl9" Sep 10 13:39:54.899748 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:54.899694 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgm9l\" (UniqueName: \"kubernetes.io/projected/be8293d4-b9ef-4d0f-9686-ae65c483f0f4-kube-api-access-bgm9l\") pod \"payload-pre-processing-e2e-shared-b-w3-89be92-6f87d675d6-jldxc\" (UID: \"be8293d4-b9ef-4d0f-9686-ae65c483f0f4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-89be92-6f87d675d6-jldxc" Sep 10 13:39:54.900191 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:54.900172 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/be8293d4-b9ef-4d0f-9686-ae65c483f0f4-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-89be92-6f87d675d6-jldxc\" (UID: \"be8293d4-b9ef-4d0f-9686-ae65c483f0f4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-89be92-6f87d675d6-jldxc" Sep 10 13:39:54.910081 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:54.910055 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgm9l\" (UniqueName: \"kubernetes.io/projected/be8293d4-b9ef-4d0f-9686-ae65c483f0f4-kube-api-access-bgm9l\") pod \"payload-pre-processing-e2e-shared-b-w3-89be92-6f87d675d6-jldxc\" (UID: \"be8293d4-b9ef-4d0f-9686-ae65c483f0f4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-89be92-6f87d675d6-jldxc" Sep 10 13:39:55.000292 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:55.000251 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tplhn\" (UniqueName: \"kubernetes.io/projected/d11b889b-57c3-4ae5-8cce-f337308aaa6d-kube-api-access-tplhn\") pod \"payload-processing-e2e-shared-b-w3-89be92-98b7b59cf-9krl9\" (UID: \"d11b889b-57c3-4ae5-8cce-f337308aaa6d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-89be92-98b7b59cf-9krl9" Sep 10 13:39:55.000504 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:55.000344 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d11b889b-57c3-4ae5-8cce-f337308aaa6d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-89be92-98b7b59cf-9krl9\" (UID: \"d11b889b-57c3-4ae5-8cce-f337308aaa6d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-89be92-98b7b59cf-9krl9" Sep 10 13:39:55.001095 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:55.001073 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d11b889b-57c3-4ae5-8cce-f337308aaa6d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-89be92-98b7b59cf-9krl9\" (UID: \"d11b889b-57c3-4ae5-8cce-f337308aaa6d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-89be92-98b7b59cf-9krl9" Sep 10 13:39:55.009847 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:55.009824 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tplhn\" (UniqueName: \"kubernetes.io/projected/d11b889b-57c3-4ae5-8cce-f337308aaa6d-kube-api-access-tplhn\") pod \"payload-processing-e2e-shared-b-w3-89be92-98b7b59cf-9krl9\" (UID: \"d11b889b-57c3-4ae5-8cce-f337308aaa6d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-89be92-98b7b59cf-9krl9" Sep 10 13:39:55.010601 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:55.010586 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-89be92-6f87d675d6-jldxc" Sep 10 13:39:55.107102 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:55.107065 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-89be92-98b7b59cf-9krl9" Sep 10 13:39:55.284594 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:55.284568 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-89be92-6f87d675d6-jldxc"] Sep 10 13:39:55.286037 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:39:55.286013 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe8293d4_b9ef_4d0f_9686_ae65c483f0f4.slice/crio-d9f8d5a77e0afd97e5c1c53348c4fe996c468d2119fcec190e1d60888b9e319b WatchSource:0}: Error finding container d9f8d5a77e0afd97e5c1c53348c4fe996c468d2119fcec190e1d60888b9e319b: Status 404 returned error can't find the container with id d9f8d5a77e0afd97e5c1c53348c4fe996c468d2119fcec190e1d60888b9e319b Sep 10 13:39:55.384245 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:55.384215 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-89be92-98b7b59cf-9krl9"] Sep 10 13:39:55.385744 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:39:55.385719 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd11b889b_57c3_4ae5_8cce_f337308aaa6d.slice/crio-1d41a967e7344dc8fbc1c448832ac0d528d2ada626e9194a705eb2e4f71d9ebc WatchSource:0}: Error finding container 1d41a967e7344dc8fbc1c448832ac0d528d2ada626e9194a705eb2e4f71d9ebc: Status 404 returned error can't find the container with id 1d41a967e7344dc8fbc1c448832ac0d528d2ada626e9194a705eb2e4f71d9ebc Sep 10 13:39:56.173141 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:56.173038 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-89be92-98b7b59cf-9krl9" event={"ID":"d11b889b-57c3-4ae5-8cce-f337308aaa6d","Type":"ContainerStarted","Data":"77e1000b202d967f1a70c8578c8958fadb70c6d0b3a4b607df96d248759ba335"} Sep 10 13:39:56.173141 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:56.173106 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-89be92-98b7b59cf-9krl9" event={"ID":"d11b889b-57c3-4ae5-8cce-f337308aaa6d","Type":"ContainerStarted","Data":"1d41a967e7344dc8fbc1c448832ac0d528d2ada626e9194a705eb2e4f71d9ebc"} Sep 10 13:39:56.173716 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:56.173232 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-89be92-98b7b59cf-9krl9" Sep 10 13:39:56.175300 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:56.175274 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-89be92-6f87d675d6-jldxc" event={"ID":"be8293d4-b9ef-4d0f-9686-ae65c483f0f4","Type":"ContainerStarted","Data":"c11c936fe3ecad3e51d1fbae5b7f87fac64267917d897e7e5875b1f55849af86"} Sep 10 13:39:56.175440 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:56.175319 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-89be92-6f87d675d6-jldxc" event={"ID":"be8293d4-b9ef-4d0f-9686-ae65c483f0f4","Type":"ContainerStarted","Data":"d9f8d5a77e0afd97e5c1c53348c4fe996c468d2119fcec190e1d60888b9e319b"} Sep 10 13:39:56.175440 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:56.175342 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-89be92-6f87d675d6-jldxc" Sep 10 13:39:56.194394 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:56.194342 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-89be92-98b7b59cf-9krl9" podStartSLOduration=2.1943258979999998 podStartE2EDuration="2.194325898s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:39:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:39:56.193089634 +0000 UTC m=+1306.087807053" watchObservedRunningTime="2026-09-10 13:39:56.194325898 +0000 UTC m=+1306.089043314" Sep 10 13:39:56.222898 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:56.222832 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-89be92-6f87d675d6-jldxc" podStartSLOduration=2.222810559 podStartE2EDuration="2.222810559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:39:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:39:56.218151768 +0000 UTC m=+1306.112869186" watchObservedRunningTime="2026-09-10 13:39:56.222810559 +0000 UTC m=+1306.117527980" Sep 10 13:39:56.455416 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:56.455306 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:56.580970 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:56.580918 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:39:57.002831 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:39:57.002761 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:40:07.184017 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:07.183975 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-89be92-6f87d675d6-jldxc" Sep 10 13:40:07.184456 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:07.184060 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-89be92-98b7b59cf-9krl9" Sep 10 13:40:07.812869 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:07.812838 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:40:07.918855 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:07.918821 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:40:08.017148 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:08.017109 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:40:08.110338 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:08.110212 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:40:08.200740 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:08.200710 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:40:08.491119 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:08.491083 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:40:15.001270 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:15.001232 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:40:16.111959 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:16.111915 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:40:16.229380 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:16.229332 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:40:16.405174 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:16.405088 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w"] Sep 10 13:40:16.405336 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:16.405295 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" podUID="6b697811-e443-42cf-bc5e-09a9919417cc" containerName="istio-proxy" containerID="cri-o://e283bfe1118fa6cefc36af1edd9eb4281fcb8b4ae012b3ee792f642af46c9b65" gracePeriod=30 Sep 10 13:40:16.436276 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:16.436237 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:40:16.768456 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:16.768389 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:40:16.860673 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:16.860594 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:40:16.987217 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:16.987164 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:40:17.255909 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:17.255839 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:40:21.655074 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:21.655049 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:40:21.752079 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:21.751983 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b697811-e443-42cf-bc5e-09a9919417cc-workload-socket\") pod \"6b697811-e443-42cf-bc5e-09a9919417cc\" (UID: \"6b697811-e443-42cf-bc5e-09a9919417cc\") " Sep 10 13:40:21.752079 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:21.752064 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frkqc\" (UniqueName: \"kubernetes.io/projected/6b697811-e443-42cf-bc5e-09a9919417cc-kube-api-access-frkqc\") pod \"6b697811-e443-42cf-bc5e-09a9919417cc\" (UID: \"6b697811-e443-42cf-bc5e-09a9919417cc\") " Sep 10 13:40:21.752338 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:21.752101 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b697811-e443-42cf-bc5e-09a9919417cc-istio-token\") pod \"6b697811-e443-42cf-bc5e-09a9919417cc\" (UID: \"6b697811-e443-42cf-bc5e-09a9919417cc\") " Sep 10 13:40:21.752338 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:21.752139 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b697811-e443-42cf-bc5e-09a9919417cc-istio-podinfo\") pod \"6b697811-e443-42cf-bc5e-09a9919417cc\" (UID: \"6b697811-e443-42cf-bc5e-09a9919417cc\") " Sep 10 13:40:21.752338 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:21.752291 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b697811-e443-42cf-bc5e-09a9919417cc-istio-envoy\") pod \"6b697811-e443-42cf-bc5e-09a9919417cc\" (UID: \"6b697811-e443-42cf-bc5e-09a9919417cc\") " Sep 10 13:40:21.752548 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:21.752338 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b697811-e443-42cf-bc5e-09a9919417cc-istiod-ca-cert\") pod \"6b697811-e443-42cf-bc5e-09a9919417cc\" (UID: \"6b697811-e443-42cf-bc5e-09a9919417cc\") " Sep 10 13:40:21.752548 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:21.752377 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b697811-e443-42cf-bc5e-09a9919417cc-istio-data\") pod \"6b697811-e443-42cf-bc5e-09a9919417cc\" (UID: \"6b697811-e443-42cf-bc5e-09a9919417cc\") " Sep 10 13:40:21.752548 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:21.752291 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b697811-e443-42cf-bc5e-09a9919417cc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6b697811-e443-42cf-bc5e-09a9919417cc" (UID: "6b697811-e443-42cf-bc5e-09a9919417cc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:40:21.752548 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:21.752460 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b697811-e443-42cf-bc5e-09a9919417cc-credential-socket\") pod \"6b697811-e443-42cf-bc5e-09a9919417cc\" (UID: \"6b697811-e443-42cf-bc5e-09a9919417cc\") " Sep 10 13:40:21.752548 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:21.752522 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b697811-e443-42cf-bc5e-09a9919417cc-workload-certs\") pod \"6b697811-e443-42cf-bc5e-09a9919417cc\" (UID: \"6b697811-e443-42cf-bc5e-09a9919417cc\") " Sep 10 13:40:21.752800 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:21.752617 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b697811-e443-42cf-bc5e-09a9919417cc-istio-data" (OuterVolumeSpecName: "istio-data") pod "6b697811-e443-42cf-bc5e-09a9919417cc" (UID: "6b697811-e443-42cf-bc5e-09a9919417cc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:40:21.752800 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:21.752718 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b697811-e443-42cf-bc5e-09a9919417cc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6b697811-e443-42cf-bc5e-09a9919417cc" (UID: "6b697811-e443-42cf-bc5e-09a9919417cc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:40:21.752906 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:21.752793 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b697811-e443-42cf-bc5e-09a9919417cc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6b697811-e443-42cf-bc5e-09a9919417cc" (UID: "6b697811-e443-42cf-bc5e-09a9919417cc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:40:21.752906 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:21.752893 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b697811-e443-42cf-bc5e-09a9919417cc-istiod-ca-cert\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:40:21.753016 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:21.752912 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b697811-e443-42cf-bc5e-09a9919417cc-istio-data\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:40:21.753016 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:21.752926 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b697811-e443-42cf-bc5e-09a9919417cc-credential-socket\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:40:21.753016 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:21.752939 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b697811-e443-42cf-bc5e-09a9919417cc-workload-socket\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:40:21.753016 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:21.752948 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b697811-e443-42cf-bc5e-09a9919417cc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6b697811-e443-42cf-bc5e-09a9919417cc" (UID: "6b697811-e443-42cf-bc5e-09a9919417cc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:40:21.754615 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:21.754583 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b697811-e443-42cf-bc5e-09a9919417cc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6b697811-e443-42cf-bc5e-09a9919417cc" (UID: "6b697811-e443-42cf-bc5e-09a9919417cc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:40:21.754615 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:21.754607 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6b697811-e443-42cf-bc5e-09a9919417cc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6b697811-e443-42cf-bc5e-09a9919417cc" (UID: "6b697811-e443-42cf-bc5e-09a9919417cc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 13:40:21.754831 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:21.754815 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b697811-e443-42cf-bc5e-09a9919417cc-istio-token" (OuterVolumeSpecName: "istio-token") pod "6b697811-e443-42cf-bc5e-09a9919417cc" (UID: "6b697811-e443-42cf-bc5e-09a9919417cc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:40:21.754891 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:21.754845 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b697811-e443-42cf-bc5e-09a9919417cc-kube-api-access-frkqc" (OuterVolumeSpecName: "kube-api-access-frkqc") pod "6b697811-e443-42cf-bc5e-09a9919417cc" (UID: "6b697811-e443-42cf-bc5e-09a9919417cc"). InnerVolumeSpecName "kube-api-access-frkqc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:40:21.853749 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:21.853714 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frkqc\" (UniqueName: \"kubernetes.io/projected/6b697811-e443-42cf-bc5e-09a9919417cc-kube-api-access-frkqc\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:40:21.853749 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:21.853743 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b697811-e443-42cf-bc5e-09a9919417cc-istio-token\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:40:21.853749 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:21.853754 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b697811-e443-42cf-bc5e-09a9919417cc-istio-podinfo\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:40:21.853989 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:21.853762 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b697811-e443-42cf-bc5e-09a9919417cc-istio-envoy\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:40:21.853989 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:21.853770 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b697811-e443-42cf-bc5e-09a9919417cc-workload-certs\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:40:22.343687 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:22.343649 2560 generic.go:356] "Generic (PLEG): container finished" podID="6b697811-e443-42cf-bc5e-09a9919417cc" containerID="e283bfe1118fa6cefc36af1edd9eb4281fcb8b4ae012b3ee792f642af46c9b65" exitCode=0 Sep 10 13:40:22.343889 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:22.343706 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" Sep 10 13:40:22.343889 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:22.343722 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" event={"ID":"6b697811-e443-42cf-bc5e-09a9919417cc","Type":"ContainerDied","Data":"e283bfe1118fa6cefc36af1edd9eb4281fcb8b4ae012b3ee792f642af46c9b65"} Sep 10 13:40:22.343889 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:22.343776 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w" event={"ID":"6b697811-e443-42cf-bc5e-09a9919417cc","Type":"ContainerDied","Data":"e2e761688ee6a250f1b1c216064a3fd0133aa4b761c96081027406b8f4d00449"} Sep 10 13:40:22.343889 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:22.343809 2560 scope.go:117] "RemoveContainer" containerID="e283bfe1118fa6cefc36af1edd9eb4281fcb8b4ae012b3ee792f642af46c9b65" Sep 10 13:40:22.354395 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:22.354377 2560 scope.go:117] "RemoveContainer" containerID="258f63a02b64c82799b7ac231f982cf1ec4c3e70785163a0e9307f2ec9d4f0e2" Sep 10 13:40:22.363612 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:22.363592 2560 scope.go:117] "RemoveContainer" containerID="e283bfe1118fa6cefc36af1edd9eb4281fcb8b4ae012b3ee792f642af46c9b65" Sep 10 13:40:22.363929 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:40:22.363909 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e283bfe1118fa6cefc36af1edd9eb4281fcb8b4ae012b3ee792f642af46c9b65\": container with ID starting with e283bfe1118fa6cefc36af1edd9eb4281fcb8b4ae012b3ee792f642af46c9b65 not found: ID does not exist" containerID="e283bfe1118fa6cefc36af1edd9eb4281fcb8b4ae012b3ee792f642af46c9b65" Sep 10 13:40:22.363995 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:22.363941 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e283bfe1118fa6cefc36af1edd9eb4281fcb8b4ae012b3ee792f642af46c9b65"} err="failed to get container status \"e283bfe1118fa6cefc36af1edd9eb4281fcb8b4ae012b3ee792f642af46c9b65\": rpc error: code = NotFound desc = could not find container \"e283bfe1118fa6cefc36af1edd9eb4281fcb8b4ae012b3ee792f642af46c9b65\": container with ID starting with e283bfe1118fa6cefc36af1edd9eb4281fcb8b4ae012b3ee792f642af46c9b65 not found: ID does not exist" Sep 10 13:40:22.363995 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:22.363965 2560 scope.go:117] "RemoveContainer" containerID="258f63a02b64c82799b7ac231f982cf1ec4c3e70785163a0e9307f2ec9d4f0e2" Sep 10 13:40:22.364220 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:40:22.364203 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"258f63a02b64c82799b7ac231f982cf1ec4c3e70785163a0e9307f2ec9d4f0e2\": container with ID starting with 258f63a02b64c82799b7ac231f982cf1ec4c3e70785163a0e9307f2ec9d4f0e2 not found: ID does not exist" containerID="258f63a02b64c82799b7ac231f982cf1ec4c3e70785163a0e9307f2ec9d4f0e2" Sep 10 13:40:22.364271 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:22.364228 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"258f63a02b64c82799b7ac231f982cf1ec4c3e70785163a0e9307f2ec9d4f0e2"} err="failed to get container status \"258f63a02b64c82799b7ac231f982cf1ec4c3e70785163a0e9307f2ec9d4f0e2\": rpc error: code = NotFound desc = could not find container \"258f63a02b64c82799b7ac231f982cf1ec4c3e70785163a0e9307f2ec9d4f0e2\": container with ID starting with 258f63a02b64c82799b7ac231f982cf1ec4c3e70785163a0e9307f2ec9d4f0e2 not found: ID does not exist" Sep 10 13:40:22.374961 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:22.374924 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w"] Sep 10 13:40:22.377331 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:22.377309 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-839f01ff-openshift-default-699846f4bd-mqq2w"] Sep 10 13:40:22.563899 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:22.563866 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b697811-e443-42cf-bc5e-09a9919417cc" path="/var/lib/kubelet/pods/6b697811-e443-42cf-bc5e-09a9919417cc/volumes" Sep 10 13:40:22.785568 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:22.785536 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:40:24.313238 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:24.313192 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:40:24.436832 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:24.436796 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:40:24.642180 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:24.642084 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:40:24.733072 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:24.733040 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:40:31.966918 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:31.966867 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:40:32.157733 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:32.157698 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:40:33.054296 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:33.054196 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:40:33.427081 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:33.427043 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:40:33.719301 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:33.719207 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:40:36.912017 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:36.911978 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:40:37.018250 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:37.018210 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:40:38.218204 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:38.218135 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:40:38.339532 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:38.339484 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:40:38.473293 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:38.473199 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:40:39.133237 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:39.133194 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:40:39.155395 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:39.155363 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5d75ac-57487949fd-djhg9"] Sep 10 13:40:39.155788 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:39.155775 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b697811-e443-42cf-bc5e-09a9919417cc" containerName="istio-proxy" Sep 10 13:40:39.155840 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:39.155790 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b697811-e443-42cf-bc5e-09a9919417cc" containerName="istio-proxy" Sep 10 13:40:39.155890 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:39.155881 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b697811-e443-42cf-bc5e-09a9919417cc" containerName="istio-proxy" Sep 10 13:40:39.155926 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:39.155894 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b697811-e443-42cf-bc5e-09a9919417cc" containerName="istio-proxy" Sep 10 13:40:39.160589 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:39.160569 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5d75ac-57487949fd-djhg9" Sep 10 13:40:39.165302 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:39.165278 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-5d75ac-dockercfg-mn8h2\"" Sep 10 13:40:39.165624 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:39.165608 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-5d75ac\"" Sep 10 13:40:39.173344 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:39.173319 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5d75ac-57487949fd-djhg9"] Sep 10 13:40:39.241201 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:39.241166 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:40:39.322239 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:39.322200 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/338e1878-2d19-4feb-8258-f03b66237bc2-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-5d75ac-57487949fd-djhg9\" (UID: \"338e1878-2d19-4feb-8258-f03b66237bc2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5d75ac-57487949fd-djhg9" Sep 10 13:40:39.322470 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:39.322263 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sx6vn\" (UniqueName: \"kubernetes.io/projected/338e1878-2d19-4feb-8258-f03b66237bc2-kube-api-access-sx6vn\") pod \"payload-pre-processing-e2e-shared-a-w2-5d75ac-57487949fd-djhg9\" (UID: \"338e1878-2d19-4feb-8258-f03b66237bc2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5d75ac-57487949fd-djhg9" Sep 10 13:40:39.423541 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:39.423504 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/338e1878-2d19-4feb-8258-f03b66237bc2-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-5d75ac-57487949fd-djhg9\" (UID: \"338e1878-2d19-4feb-8258-f03b66237bc2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5d75ac-57487949fd-djhg9" Sep 10 13:40:39.423722 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:39.423585 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sx6vn\" (UniqueName: \"kubernetes.io/projected/338e1878-2d19-4feb-8258-f03b66237bc2-kube-api-access-sx6vn\") pod \"payload-pre-processing-e2e-shared-a-w2-5d75ac-57487949fd-djhg9\" (UID: \"338e1878-2d19-4feb-8258-f03b66237bc2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5d75ac-57487949fd-djhg9" Sep 10 13:40:39.424239 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:39.424217 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/338e1878-2d19-4feb-8258-f03b66237bc2-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-5d75ac-57487949fd-djhg9\" (UID: \"338e1878-2d19-4feb-8258-f03b66237bc2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5d75ac-57487949fd-djhg9" Sep 10 13:40:39.435052 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:39.435023 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sx6vn\" (UniqueName: \"kubernetes.io/projected/338e1878-2d19-4feb-8258-f03b66237bc2-kube-api-access-sx6vn\") pod \"payload-pre-processing-e2e-shared-a-w2-5d75ac-57487949fd-djhg9\" (UID: \"338e1878-2d19-4feb-8258-f03b66237bc2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5d75ac-57487949fd-djhg9" Sep 10 13:40:39.470805 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:39.470761 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5d75ac-57487949fd-djhg9" Sep 10 13:40:39.754279 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:39.754250 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5d75ac-57487949fd-djhg9"] Sep 10 13:40:39.756255 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:40:39.756228 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod338e1878_2d19_4feb_8258_f03b66237bc2.slice/crio-08e37360227a53b8b6ee54370398cea483a7ae7907a048499d2e0d0cb53135e4 WatchSource:0}: Error finding container 08e37360227a53b8b6ee54370398cea483a7ae7907a048499d2e0d0cb53135e4: Status 404 returned error can't find the container with id 08e37360227a53b8b6ee54370398cea483a7ae7907a048499d2e0d0cb53135e4 Sep 10 13:40:40.449907 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:40.449866 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5d75ac-57487949fd-djhg9" event={"ID":"338e1878-2d19-4feb-8258-f03b66237bc2","Type":"ContainerStarted","Data":"6e9562f49c15cd10303e59de13f93258dd5d5b7ca29f9391a560729debba5979"} Sep 10 13:40:40.450300 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:40.449929 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5d75ac-57487949fd-djhg9" event={"ID":"338e1878-2d19-4feb-8258-f03b66237bc2","Type":"ContainerStarted","Data":"08e37360227a53b8b6ee54370398cea483a7ae7907a048499d2e0d0cb53135e4"} Sep 10 13:40:40.450300 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:40.450055 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5d75ac-57487949fd-djhg9" Sep 10 13:40:40.505573 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:40.505519 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5d75ac-57487949fd-djhg9" podStartSLOduration=1.5055020460000001 podStartE2EDuration="1.505502046s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:40:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:40:40.503031809 +0000 UTC m=+1350.397749227" watchObservedRunningTime="2026-09-10 13:40:40.505502046 +0000 UTC m=+1350.400219598" Sep 10 13:40:51.458025 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:40:51.457989 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5d75ac-57487949fd-djhg9" Sep 10 13:41:01.911160 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:01.911116 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:41:02.251054 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:02.251021 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:41:02.570420 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:02.570320 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj"] Sep 10 13:41:02.571597 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:02.571573 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b697811-e443-42cf-bc5e-09a9919417cc" containerName="istio-proxy" Sep 10 13:41:02.571748 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:02.571736 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b697811-e443-42cf-bc5e-09a9919417cc" containerName="istio-proxy" Sep 10 13:41:02.576611 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:02.576587 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:41:02.580311 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:02.580286 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-ba174e-openshift-default-dockercfg-69vvs\"" Sep 10 13:41:02.589860 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:02.589824 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj"] Sep 10 13:41:02.627071 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:02.627032 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:41:02.631840 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:02.631815 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-istio-envoy\") pod \"e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj\" (UID: \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\") " pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:41:02.631960 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:02.631861 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-credential-socket\") pod \"e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj\" (UID: \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\") " pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:41:02.631960 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:02.631919 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-workload-certs\") pod \"e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj\" (UID: \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\") " pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:41:02.631960 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:02.631944 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-workload-socket\") pod \"e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj\" (UID: \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\") " pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:41:02.632096 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:02.631965 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-istio-podinfo\") pod \"e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj\" (UID: \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\") " pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:41:02.632096 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:02.631991 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-istio-data\") pod \"e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj\" (UID: \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\") " pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:41:02.632096 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:02.632045 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-istio-token\") pod \"e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj\" (UID: \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\") " pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:41:02.632211 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:02.632096 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzhsr\" (UniqueName: \"kubernetes.io/projected/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-kube-api-access-xzhsr\") pod \"e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj\" (UID: \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\") " pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:41:02.632211 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:02.632170 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-istiod-ca-cert\") pod \"e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj\" (UID: \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\") " pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:41:02.733511 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:02.733463 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-istio-envoy\") pod \"e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj\" (UID: \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\") " pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:41:02.733715 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:02.733522 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-credential-socket\") pod \"e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj\" (UID: \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\") " pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:41:02.733715 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:02.733563 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-workload-certs\") pod \"e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj\" (UID: \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\") " pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:41:02.733715 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:02.733588 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-workload-socket\") pod \"e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj\" (UID: \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\") " pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:41:02.733715 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:02.733611 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-istio-podinfo\") pod \"e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj\" (UID: \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\") " pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:41:02.733715 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:02.733645 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-istio-data\") pod \"e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj\" (UID: \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\") " pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:41:02.733715 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:02.733695 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-istio-token\") pod \"e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj\" (UID: \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\") " pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:41:02.734054 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:02.733727 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzhsr\" (UniqueName: \"kubernetes.io/projected/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-kube-api-access-xzhsr\") pod \"e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj\" (UID: \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\") " pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:41:02.734054 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:02.733777 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-istiod-ca-cert\") pod \"e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj\" (UID: \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\") " pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:41:02.734054 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:02.734006 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-credential-socket\") pod \"e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj\" (UID: \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\") " pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:41:02.734054 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:02.734030 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-workload-certs\") pod \"e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj\" (UID: \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\") " pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:41:02.734357 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:02.734332 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-istio-data\") pod \"e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj\" (UID: \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\") " pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:41:02.734455 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:02.734439 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-workload-socket\") pod \"e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj\" (UID: \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\") " pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:41:02.734668 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:02.734648 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-istiod-ca-cert\") pod \"e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj\" (UID: \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\") " pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:41:02.736156 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:02.736138 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-istio-envoy\") pod \"e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj\" (UID: \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\") " pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:41:02.736238 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:02.736208 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-istio-podinfo\") pod \"e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj\" (UID: \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\") " pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:41:02.744784 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:02.744758 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-istio-token\") pod \"e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj\" (UID: \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\") " pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:41:02.744954 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:02.744893 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzhsr\" (UniqueName: \"kubernetes.io/projected/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-kube-api-access-xzhsr\") pod \"e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj\" (UID: \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\") " pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:41:02.851976 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:02.851874 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:41:02.890764 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:02.890720 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:41:03.185811 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:03.185784 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj"] Sep 10 13:41:03.187772 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:41:03.187741 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a1f48da_7c79_4e1f_a9e5_6371c4445a38.slice/crio-08cc1f76563116dc8fda1f784728feeb4c864e7fff5d06af5af8e71f29f31f36 WatchSource:0}: Error finding container 08cc1f76563116dc8fda1f784728feeb4c864e7fff5d06af5af8e71f29f31f36: Status 404 returned error can't find the container with id 08cc1f76563116dc8fda1f784728feeb4c864e7fff5d06af5af8e71f29f31f36 Sep 10 13:41:03.189954 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:03.189921 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:41:03.190056 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:03.190009 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:41:03.190122 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:03.190054 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:41:03.598941 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:03.598846 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" event={"ID":"6a1f48da-7c79-4e1f-a9e5-6371c4445a38","Type":"ContainerStarted","Data":"0605f2a309df51b17a2b06717bc2c267deb2bc96b2161554bae5abf725d5c07f"} Sep 10 13:41:03.598941 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:03.598899 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" event={"ID":"6a1f48da-7c79-4e1f-a9e5-6371c4445a38","Type":"ContainerStarted","Data":"08cc1f76563116dc8fda1f784728feeb4c864e7fff5d06af5af8e71f29f31f36"} Sep 10 13:41:03.625918 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:03.625860 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" podStartSLOduration=1.625841956 podStartE2EDuration="1.625841956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:41:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:41:03.623717351 +0000 UTC m=+1373.518434764" watchObservedRunningTime="2026-09-10 13:41:03.625841956 +0000 UTC m=+1373.520559370" Sep 10 13:41:03.890945 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:03.890857 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:41:04.897174 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:04.897138 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:41:05.586660 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:05.586621 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:41:05.611077 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:05.611047 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:41:05.612155 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:05.612132 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:41:07.437875 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:07.437837 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:41:07.762947 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:07.762836 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:41:07.898102 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:07.898069 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:41:07.966719 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:07.966683 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ba174e-7c68c6657d-7l78v"] Sep 10 13:41:07.970539 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:07.970521 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ba174e-7c68c6657d-7l78v" Sep 10 13:41:07.974139 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:07.974117 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-ba174e-dockercfg-nkx64\"" Sep 10 13:41:07.974726 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:07.974706 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-ba174e\"" Sep 10 13:41:07.983335 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:07.983304 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ba174e-7c68c6657d-7l78v"] Sep 10 13:41:08.084974 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:08.084883 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88a911a9-b2dc-400f-b27c-4cca66213ef9-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-ba174e-7c68c6657d-7l78v\" (UID: \"88a911a9-b2dc-400f-b27c-4cca66213ef9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ba174e-7c68c6657d-7l78v" Sep 10 13:41:08.085154 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:08.085003 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wc5c\" (UniqueName: \"kubernetes.io/projected/88a911a9-b2dc-400f-b27c-4cca66213ef9-kube-api-access-5wc5c\") pod \"payload-pre-processing-e2e-shared-b-w2-ba174e-7c68c6657d-7l78v\" (UID: \"88a911a9-b2dc-400f-b27c-4cca66213ef9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ba174e-7c68c6657d-7l78v" Sep 10 13:41:08.186251 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:08.186204 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wc5c\" (UniqueName: \"kubernetes.io/projected/88a911a9-b2dc-400f-b27c-4cca66213ef9-kube-api-access-5wc5c\") pod \"payload-pre-processing-e2e-shared-b-w2-ba174e-7c68c6657d-7l78v\" (UID: \"88a911a9-b2dc-400f-b27c-4cca66213ef9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ba174e-7c68c6657d-7l78v" Sep 10 13:41:08.186473 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:08.186338 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88a911a9-b2dc-400f-b27c-4cca66213ef9-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-ba174e-7c68c6657d-7l78v\" (UID: \"88a911a9-b2dc-400f-b27c-4cca66213ef9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ba174e-7c68c6657d-7l78v" Sep 10 13:41:08.187094 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:08.187074 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88a911a9-b2dc-400f-b27c-4cca66213ef9-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-ba174e-7c68c6657d-7l78v\" (UID: \"88a911a9-b2dc-400f-b27c-4cca66213ef9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ba174e-7c68c6657d-7l78v" Sep 10 13:41:08.203244 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:08.203212 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wc5c\" (UniqueName: \"kubernetes.io/projected/88a911a9-b2dc-400f-b27c-4cca66213ef9-kube-api-access-5wc5c\") pod \"payload-pre-processing-e2e-shared-b-w2-ba174e-7c68c6657d-7l78v\" (UID: \"88a911a9-b2dc-400f-b27c-4cca66213ef9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ba174e-7c68c6657d-7l78v" Sep 10 13:41:08.282704 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:08.282661 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ba174e-7c68c6657d-7l78v" Sep 10 13:41:08.583241 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:08.583212 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ba174e-7c68c6657d-7l78v"] Sep 10 13:41:08.585439 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:41:08.585396 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88a911a9_b2dc_400f_b27c_4cca66213ef9.slice/crio-04cea5d0abf73c7aad315098e4731348085ac53259ec4558cc75e7047b62f076 WatchSource:0}: Error finding container 04cea5d0abf73c7aad315098e4731348085ac53259ec4558cc75e7047b62f076: Status 404 returned error can't find the container with id 04cea5d0abf73c7aad315098e4731348085ac53259ec4558cc75e7047b62f076 Sep 10 13:41:08.634263 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:08.634230 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ba174e-7c68c6657d-7l78v" event={"ID":"88a911a9-b2dc-400f-b27c-4cca66213ef9","Type":"ContainerStarted","Data":"04cea5d0abf73c7aad315098e4731348085ac53259ec4558cc75e7047b62f076"} Sep 10 13:41:09.642175 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:09.642140 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ba174e-7c68c6657d-7l78v" event={"ID":"88a911a9-b2dc-400f-b27c-4cca66213ef9","Type":"ContainerStarted","Data":"8ca8122feda20997afaaa9d333064d8d2050b244215ce6956e4951db1a5f0f05"} Sep 10 13:41:09.642588 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:09.642312 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ba174e-7c68c6657d-7l78v" Sep 10 13:41:09.665685 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:09.665629 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ba174e-7c68c6657d-7l78v" podStartSLOduration=2.665611713 podStartE2EDuration="2.665611713s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:41:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:41:09.662539123 +0000 UTC m=+1379.557256541" watchObservedRunningTime="2026-09-10 13:41:09.665611713 +0000 UTC m=+1379.560329131" Sep 10 13:41:20.660614 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:20.660579 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ba174e-7c68c6657d-7l78v" Sep 10 13:41:25.933415 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:25.933367 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:41:27.752174 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:27.752137 2560 generic.go:356] "Generic (PLEG): container finished" podID="6a1f48da-7c79-4e1f-a9e5-6371c4445a38" containerID="0605f2a309df51b17a2b06717bc2c267deb2bc96b2161554bae5abf725d5c07f" exitCode=0 Sep 10 13:41:27.752611 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:27.752211 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" event={"ID":"6a1f48da-7c79-4e1f-a9e5-6371c4445a38","Type":"ContainerDied","Data":"0605f2a309df51b17a2b06717bc2c267deb2bc96b2161554bae5abf725d5c07f"} Sep 10 13:41:27.752731 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:27.752714 2560 scope.go:117] "RemoveContainer" containerID="0605f2a309df51b17a2b06717bc2c267deb2bc96b2161554bae5abf725d5c07f" Sep 10 13:41:27.753979 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:27.753953 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:41:27.754046 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:27.754020 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:41:27.754083 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:27.754054 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:41:28.357114 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:28.357077 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:41:28.779671 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:28.779603 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" event={"ID":"6a1f48da-7c79-4e1f-a9e5-6371c4445a38","Type":"ContainerStarted","Data":"abe819dba1436d392d6a02e30550282b7172d4cbb9c62b681e068c2793779e9d"} Sep 10 13:41:28.891264 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:28.891217 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:41:28.891507 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:28.891486 2560 patch_prober.go:28] interesting pod/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Sep 10 13:41:28.891571 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:28.891534 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" podUID="6a1f48da-7c79-4e1f-a9e5-6371c4445a38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Sep 10 13:41:29.891652 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:29.891613 2560 patch_prober.go:28] interesting pod/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Sep 10 13:41:29.892066 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:29.891697 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" podUID="6a1f48da-7c79-4e1f-a9e5-6371c4445a38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Sep 10 13:41:30.879837 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:30.879794 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:41:30.891145 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:30.891099 2560 patch_prober.go:28] interesting pod/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Sep 10 13:41:30.891329 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:30.891163 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" podUID="6a1f48da-7c79-4e1f-a9e5-6371c4445a38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Sep 10 13:41:31.892052 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:31.892014 2560 patch_prober.go:28] interesting pod/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Sep 10 13:41:31.892481 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:31.892071 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" podUID="6a1f48da-7c79-4e1f-a9e5-6371c4445a38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Sep 10 13:41:32.891724 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:32.891684 2560 patch_prober.go:28] interesting pod/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Sep 10 13:41:32.892092 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:32.891770 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" podUID="6a1f48da-7c79-4e1f-a9e5-6371c4445a38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Sep 10 13:41:33.891451 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:33.891394 2560 patch_prober.go:28] interesting pod/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Sep 10 13:41:33.891643 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:33.891468 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" podUID="6a1f48da-7c79-4e1f-a9e5-6371c4445a38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Sep 10 13:41:34.895768 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:34.895734 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:41:34.896221 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:34.896146 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:41:34.896968 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:34.896949 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:41:38.862152 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:38.862096 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:41:38.942243 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:38.942203 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:41:40.123754 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:40.123717 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:41:40.425448 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:40.425388 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:41:41.054471 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:41.054396 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:41:44.102828 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:44.102789 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:41:44.214022 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:44.213993 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:41:45.395309 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:45.395268 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:41:45.496575 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:45.496516 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:41:45.624624 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:45.624585 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:41:46.007015 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:46.006977 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:41:46.104243 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:46.104206 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:41:46.138804 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:46.138769 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9469c838-5568c6bb77-t5js8"] Sep 10 13:41:46.142535 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:46.142514 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9469c838-5568c6bb77-t5js8" Sep 10 13:41:46.145042 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:46.145023 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-9469c838-dockercfg-lg8j4\"" Sep 10 13:41:46.145121 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:46.145020 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-9469c838\"" Sep 10 13:41:46.154842 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:46.154812 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9469c838-5568c6bb77-t5js8"] Sep 10 13:41:46.160991 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:46.160963 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb09abf0-11a7-48a7-9138-015aaa58002b-plugins-config-volume\") pod \"payload-processing-e2e-mt-9469c838-5568c6bb77-t5js8\" (UID: \"bb09abf0-11a7-48a7-9138-015aaa58002b\") " pod="openshift-ingress/payload-processing-e2e-mt-9469c838-5568c6bb77-t5js8" Sep 10 13:41:46.161137 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:46.160997 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rfhs\" (UniqueName: \"kubernetes.io/projected/bb09abf0-11a7-48a7-9138-015aaa58002b-kube-api-access-6rfhs\") pod \"payload-processing-e2e-mt-9469c838-5568c6bb77-t5js8\" (UID: \"bb09abf0-11a7-48a7-9138-015aaa58002b\") " pod="openshift-ingress/payload-processing-e2e-mt-9469c838-5568c6bb77-t5js8" Sep 10 13:41:46.189812 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:46.189773 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9469c838-6645f56774-lpvp6"] Sep 10 13:41:46.193310 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:46.193291 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9469c838-6645f56774-lpvp6" Sep 10 13:41:46.207902 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:46.207867 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9469c838-6645f56774-lpvp6"] Sep 10 13:41:46.262177 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:46.262069 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb09abf0-11a7-48a7-9138-015aaa58002b-plugins-config-volume\") pod \"payload-processing-e2e-mt-9469c838-5568c6bb77-t5js8\" (UID: \"bb09abf0-11a7-48a7-9138-015aaa58002b\") " pod="openshift-ingress/payload-processing-e2e-mt-9469c838-5568c6bb77-t5js8" Sep 10 13:41:46.262351 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:46.262195 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rfhs\" (UniqueName: \"kubernetes.io/projected/bb09abf0-11a7-48a7-9138-015aaa58002b-kube-api-access-6rfhs\") pod \"payload-processing-e2e-mt-9469c838-5568c6bb77-t5js8\" (UID: \"bb09abf0-11a7-48a7-9138-015aaa58002b\") " pod="openshift-ingress/payload-processing-e2e-mt-9469c838-5568c6bb77-t5js8" Sep 10 13:41:46.262351 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:46.262265 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebacc7db-cacd-4d21-a123-d087c763037c-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-9469c838-6645f56774-lpvp6\" (UID: \"ebacc7db-cacd-4d21-a123-d087c763037c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9469c838-6645f56774-lpvp6" Sep 10 13:41:46.262351 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:46.262334 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwjmv\" (UniqueName: \"kubernetes.io/projected/ebacc7db-cacd-4d21-a123-d087c763037c-kube-api-access-cwjmv\") pod \"payload-pre-processing-e2e-mt-9469c838-6645f56774-lpvp6\" (UID: \"ebacc7db-cacd-4d21-a123-d087c763037c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9469c838-6645f56774-lpvp6" Sep 10 13:41:46.263085 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:46.263052 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb09abf0-11a7-48a7-9138-015aaa58002b-plugins-config-volume\") pod \"payload-processing-e2e-mt-9469c838-5568c6bb77-t5js8\" (UID: \"bb09abf0-11a7-48a7-9138-015aaa58002b\") " pod="openshift-ingress/payload-processing-e2e-mt-9469c838-5568c6bb77-t5js8" Sep 10 13:41:46.280542 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:46.280508 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rfhs\" (UniqueName: \"kubernetes.io/projected/bb09abf0-11a7-48a7-9138-015aaa58002b-kube-api-access-6rfhs\") pod \"payload-processing-e2e-mt-9469c838-5568c6bb77-t5js8\" (UID: \"bb09abf0-11a7-48a7-9138-015aaa58002b\") " pod="openshift-ingress/payload-processing-e2e-mt-9469c838-5568c6bb77-t5js8" Sep 10 13:41:46.363635 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:46.363592 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebacc7db-cacd-4d21-a123-d087c763037c-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-9469c838-6645f56774-lpvp6\" (UID: \"ebacc7db-cacd-4d21-a123-d087c763037c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9469c838-6645f56774-lpvp6" Sep 10 13:41:46.363819 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:46.363665 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwjmv\" (UniqueName: \"kubernetes.io/projected/ebacc7db-cacd-4d21-a123-d087c763037c-kube-api-access-cwjmv\") pod \"payload-pre-processing-e2e-mt-9469c838-6645f56774-lpvp6\" (UID: \"ebacc7db-cacd-4d21-a123-d087c763037c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9469c838-6645f56774-lpvp6" Sep 10 13:41:46.364331 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:46.364312 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebacc7db-cacd-4d21-a123-d087c763037c-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-9469c838-6645f56774-lpvp6\" (UID: \"ebacc7db-cacd-4d21-a123-d087c763037c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9469c838-6645f56774-lpvp6" Sep 10 13:41:46.373756 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:46.373721 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwjmv\" (UniqueName: \"kubernetes.io/projected/ebacc7db-cacd-4d21-a123-d087c763037c-kube-api-access-cwjmv\") pod \"payload-pre-processing-e2e-mt-9469c838-6645f56774-lpvp6\" (UID: \"ebacc7db-cacd-4d21-a123-d087c763037c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9469c838-6645f56774-lpvp6" Sep 10 13:41:46.454004 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:46.453965 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9469c838-5568c6bb77-t5js8" Sep 10 13:41:46.504664 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:46.504384 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9469c838-6645f56774-lpvp6" Sep 10 13:41:46.742686 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:46.742659 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9469c838-5568c6bb77-t5js8"] Sep 10 13:41:46.745321 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:41:46.745295 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb09abf0_11a7_48a7_9138_015aaa58002b.slice/crio-c7c2a3538751e92ae28f8ebc069eca7b7816016363105d415594b487b87bbe2e WatchSource:0}: Error finding container c7c2a3538751e92ae28f8ebc069eca7b7816016363105d415594b487b87bbe2e: Status 404 returned error can't find the container with id c7c2a3538751e92ae28f8ebc069eca7b7816016363105d415594b487b87bbe2e Sep 10 13:41:46.798270 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:46.798247 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9469c838-6645f56774-lpvp6"] Sep 10 13:41:46.799009 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:41:46.798985 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebacc7db_cacd_4d21_a123_d087c763037c.slice/crio-4cd9afe5ad55752ac7ee463ed86a149641c66c21b0d41d499094748efd7598ac WatchSource:0}: Error finding container 4cd9afe5ad55752ac7ee463ed86a149641c66c21b0d41d499094748efd7598ac: Status 404 returned error can't find the container with id 4cd9afe5ad55752ac7ee463ed86a149641c66c21b0d41d499094748efd7598ac Sep 10 13:41:46.890638 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:46.890607 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9469c838-6645f56774-lpvp6" event={"ID":"ebacc7db-cacd-4d21-a123-d087c763037c","Type":"ContainerStarted","Data":"a923b35941d98881154b1592a70f261019571175e3ee9d64645fdbea2222c8da"} Sep 10 13:41:46.890848 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:46.890669 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9469c838-6645f56774-lpvp6" event={"ID":"ebacc7db-cacd-4d21-a123-d087c763037c","Type":"ContainerStarted","Data":"4cd9afe5ad55752ac7ee463ed86a149641c66c21b0d41d499094748efd7598ac"} Sep 10 13:41:46.890848 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:46.890732 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-9469c838-6645f56774-lpvp6" Sep 10 13:41:46.892549 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:46.892523 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9469c838-5568c6bb77-t5js8" event={"ID":"bb09abf0-11a7-48a7-9138-015aaa58002b","Type":"ContainerStarted","Data":"6a76f5dc1c1f63ce0bab89ae96792b5869e34a5345c99f6f584f8a6497b36c4c"} Sep 10 13:41:46.892698 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:46.892572 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9469c838-5568c6bb77-t5js8" event={"ID":"bb09abf0-11a7-48a7-9138-015aaa58002b","Type":"ContainerStarted","Data":"c7c2a3538751e92ae28f8ebc069eca7b7816016363105d415594b487b87bbe2e"} Sep 10 13:41:46.892698 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:46.892602 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-9469c838-5568c6bb77-t5js8" Sep 10 13:41:46.914781 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:46.914720 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-9469c838-6645f56774-lpvp6" podStartSLOduration=0.914698778 podStartE2EDuration="914.698778ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:41:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:41:46.910293829 +0000 UTC m=+1416.805011249" watchObservedRunningTime="2026-09-10 13:41:46.914698778 +0000 UTC m=+1416.809416196" Sep 10 13:41:46.929517 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:46.929457 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-9469c838-5568c6bb77-t5js8" podStartSLOduration=0.929436078 podStartE2EDuration="929.436078ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:41:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:41:46.929348957 +0000 UTC m=+1416.824066376" watchObservedRunningTime="2026-09-10 13:41:46.929436078 +0000 UTC m=+1416.824153497" Sep 10 13:41:57.905881 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:57.905848 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-9469c838-6645f56774-lpvp6" Sep 10 13:41:57.906334 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:41:57.906039 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-9469c838-5568c6bb77-t5js8" Sep 10 13:42:09.265029 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:42:09.264988 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:42:10.703846 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:42:10.703812 2560 scope.go:117] "RemoveContainer" containerID="60a43423c52c206bc4400abd8c00950f5bf32149f80f8618e4b85542795e3eb6" Sep 10 13:42:14.602678 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:42:14.602641 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:42:45.909277 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:42:45.909239 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:42:46.009051 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:42:46.009016 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:42:59.145084 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:42:59.145049 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:42:59.174789 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:42:59.174756 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9469c838-5568c6bb77-t5js8"] Sep 10 13:42:59.175017 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:42:59.174989 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-9469c838-5568c6bb77-t5js8" podUID="bb09abf0-11a7-48a7-9138-015aaa58002b" containerName="payload-processing" containerID="cri-o://6a76f5dc1c1f63ce0bab89ae96792b5869e34a5345c99f6f584f8a6497b36c4c" gracePeriod=30 Sep 10 13:42:59.187707 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:42:59.187670 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9469c838-6645f56774-lpvp6"] Sep 10 13:42:59.187968 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:42:59.187936 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-9469c838-6645f56774-lpvp6" podUID="ebacc7db-cacd-4d21-a123-d087c763037c" containerName="payload-pre-processing" containerID="cri-o://a923b35941d98881154b1592a70f261019571175e3ee9d64645fdbea2222c8da" gracePeriod=30 Sep 10 13:42:59.225126 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:42:59.225092 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:42:59.298018 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:42:59.297988 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:42:59.343804 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:42:59.343689 2560 generic.go:356] "Generic (PLEG): container finished" podID="ebacc7db-cacd-4d21-a123-d087c763037c" containerID="a923b35941d98881154b1592a70f261019571175e3ee9d64645fdbea2222c8da" exitCode=0 Sep 10 13:42:59.343804 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:42:59.343714 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9469c838-6645f56774-lpvp6" event={"ID":"ebacc7db-cacd-4d21-a123-d087c763037c","Type":"ContainerDied","Data":"a923b35941d98881154b1592a70f261019571175e3ee9d64645fdbea2222c8da"} Sep 10 13:42:59.345447 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:42:59.345426 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9469c838-5568c6bb77-t5js8" event={"ID":"bb09abf0-11a7-48a7-9138-015aaa58002b","Type":"ContainerDied","Data":"6a76f5dc1c1f63ce0bab89ae96792b5869e34a5345c99f6f584f8a6497b36c4c"} Sep 10 13:42:59.345557 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:42:59.345423 2560 generic.go:356] "Generic (PLEG): container finished" podID="bb09abf0-11a7-48a7-9138-015aaa58002b" containerID="6a76f5dc1c1f63ce0bab89ae96792b5869e34a5345c99f6f584f8a6497b36c4c" exitCode=0 Sep 10 13:42:59.383611 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:42:59.383571 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:42:59.449951 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:42:59.449926 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9469c838-5568c6bb77-t5js8" Sep 10 13:42:59.479497 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:42:59.479470 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9469c838-6645f56774-lpvp6" Sep 10 13:42:59.510169 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:42:59.509188 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb09abf0-11a7-48a7-9138-015aaa58002b-plugins-config-volume\") pod \"bb09abf0-11a7-48a7-9138-015aaa58002b\" (UID: \"bb09abf0-11a7-48a7-9138-015aaa58002b\") " Sep 10 13:42:59.510169 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:42:59.509369 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6rfhs\" (UniqueName: \"kubernetes.io/projected/bb09abf0-11a7-48a7-9138-015aaa58002b-kube-api-access-6rfhs\") pod \"bb09abf0-11a7-48a7-9138-015aaa58002b\" (UID: \"bb09abf0-11a7-48a7-9138-015aaa58002b\") " Sep 10 13:42:59.511259 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:42:59.511230 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb09abf0-11a7-48a7-9138-015aaa58002b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bb09abf0-11a7-48a7-9138-015aaa58002b" (UID: "bb09abf0-11a7-48a7-9138-015aaa58002b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:42:59.513961 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:42:59.513930 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb09abf0-11a7-48a7-9138-015aaa58002b-kube-api-access-6rfhs" (OuterVolumeSpecName: "kube-api-access-6rfhs") pod "bb09abf0-11a7-48a7-9138-015aaa58002b" (UID: "bb09abf0-11a7-48a7-9138-015aaa58002b"). InnerVolumeSpecName "kube-api-access-6rfhs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:42:59.554206 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:42:59.554161 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:42:59.610448 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:42:59.610378 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cwjmv\" (UniqueName: \"kubernetes.io/projected/ebacc7db-cacd-4d21-a123-d087c763037c-kube-api-access-cwjmv\") pod \"ebacc7db-cacd-4d21-a123-d087c763037c\" (UID: \"ebacc7db-cacd-4d21-a123-d087c763037c\") " Sep 10 13:42:59.610662 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:42:59.610542 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebacc7db-cacd-4d21-a123-d087c763037c-plugins-config-volume\") pod \"ebacc7db-cacd-4d21-a123-d087c763037c\" (UID: \"ebacc7db-cacd-4d21-a123-d087c763037c\") " Sep 10 13:42:59.610841 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:42:59.610822 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rfhs\" (UniqueName: \"kubernetes.io/projected/bb09abf0-11a7-48a7-9138-015aaa58002b-kube-api-access-6rfhs\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:42:59.610904 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:42:59.610846 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb09abf0-11a7-48a7-9138-015aaa58002b-plugins-config-volume\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:42:59.610904 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:42:59.610874 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebacc7db-cacd-4d21-a123-d087c763037c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ebacc7db-cacd-4d21-a123-d087c763037c" (UID: "ebacc7db-cacd-4d21-a123-d087c763037c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:42:59.612461 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:42:59.612441 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebacc7db-cacd-4d21-a123-d087c763037c-kube-api-access-cwjmv" (OuterVolumeSpecName: "kube-api-access-cwjmv") pod "ebacc7db-cacd-4d21-a123-d087c763037c" (UID: "ebacc7db-cacd-4d21-a123-d087c763037c"). InnerVolumeSpecName "kube-api-access-cwjmv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:42:59.711981 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:42:59.711948 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwjmv\" (UniqueName: \"kubernetes.io/projected/ebacc7db-cacd-4d21-a123-d087c763037c-kube-api-access-cwjmv\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:42:59.711981 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:42:59.711978 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebacc7db-cacd-4d21-a123-d087c763037c-plugins-config-volume\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:43:00.352831 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:43:00.352799 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9469c838-6645f56774-lpvp6" Sep 10 13:43:00.353278 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:43:00.352829 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9469c838-6645f56774-lpvp6" event={"ID":"ebacc7db-cacd-4d21-a123-d087c763037c","Type":"ContainerDied","Data":"4cd9afe5ad55752ac7ee463ed86a149641c66c21b0d41d499094748efd7598ac"} Sep 10 13:43:00.353278 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:43:00.352889 2560 scope.go:117] "RemoveContainer" containerID="a923b35941d98881154b1592a70f261019571175e3ee9d64645fdbea2222c8da" Sep 10 13:43:00.354292 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:43:00.354269 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9469c838-5568c6bb77-t5js8" Sep 10 13:43:00.354362 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:43:00.354308 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9469c838-5568c6bb77-t5js8" event={"ID":"bb09abf0-11a7-48a7-9138-015aaa58002b","Type":"ContainerDied","Data":"c7c2a3538751e92ae28f8ebc069eca7b7816016363105d415594b487b87bbe2e"} Sep 10 13:43:00.366538 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:43:00.366419 2560 scope.go:117] "RemoveContainer" containerID="6a76f5dc1c1f63ce0bab89ae96792b5869e34a5345c99f6f584f8a6497b36c4c" Sep 10 13:43:00.384101 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:43:00.384067 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9469c838-5568c6bb77-t5js8"] Sep 10 13:43:00.388462 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:43:00.388436 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9469c838-5568c6bb77-t5js8"] Sep 10 13:43:00.401512 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:43:00.401482 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9469c838-6645f56774-lpvp6"] Sep 10 13:43:00.406813 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:43:00.406788 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9469c838-6645f56774-lpvp6"] Sep 10 13:43:00.563714 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:43:00.563635 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb09abf0-11a7-48a7-9138-015aaa58002b" path="/var/lib/kubelet/pods/bb09abf0-11a7-48a7-9138-015aaa58002b/volumes" Sep 10 13:43:00.563993 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:43:00.563980 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebacc7db-cacd-4d21-a123-d087c763037c" path="/var/lib/kubelet/pods/ebacc7db-cacd-4d21-a123-d087c763037c/volumes" Sep 10 13:43:02.544265 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:43:02.544232 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:43:07.617203 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:43:07.617162 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:43:09.190868 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:43:09.190825 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:43:10.598653 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:43:10.598627 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gtqlh_52fceedd-00f2-4943-ae47-dec51216d8e4/console-operator/2.log" Sep 10 13:43:10.603706 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:43:10.603682 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gtqlh_52fceedd-00f2-4943-ae47-dec51216d8e4/console-operator/2.log" Sep 10 13:43:12.447995 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:43:12.447962 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:43:13.643223 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:43:13.643189 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:43:31.617960 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:43:31.617927 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:43:44.947576 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:43:44.947546 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:43:45.100742 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:43:45.100707 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:43:51.177145 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:43:51.177105 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:43:51.281245 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:43:51.281213 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:43:51.512708 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:43:51.512612 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:43:52.875322 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:43:52.875292 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:43:52.962936 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:43:52.962906 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:43:53.063582 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:43:53.063546 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:43:53.478519 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:43:53.478483 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:43:53.844236 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:43:53.844150 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:43:53.941670 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:43:53.941637 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:43:55.324822 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:43:55.324789 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:44:01.245663 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:01.245629 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:44:01.344210 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:01.344175 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:44:01.445422 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:01.445368 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:44:01.544984 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:01.544901 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:44:02.149927 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:02.149894 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:44:08.756051 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:08.756015 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:44:08.860235 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:08.860199 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:44:08.956219 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:08.956174 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:44:09.058815 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:09.058733 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:44:19.545529 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:19.545474 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:44:19.773949 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:19.773915 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:44:27.496596 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:27.496529 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:44:40.759940 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:40.759902 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:44:46.696665 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:46.696632 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:44:47.903056 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:47.903022 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:44:48.005364 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:48.005328 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:44:48.247925 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:48.247885 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:44:48.470992 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:48.470961 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:44:48.580339 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:48.580243 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:44:48.836485 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:48.836364 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:44:49.378048 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:49.378011 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:44:51.781272 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:51.781219 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:44:51.878804 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:51.878752 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:44:52.941729 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:52.941697 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:44:53.009962 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:53.009930 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:44:53.255034 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:53.254941 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:44:53.456961 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:53.456931 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:44:53.557369 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:53.557282 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:44:53.585626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:53.585594 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a2d945-c7677fc7c-b8jdd"] Sep 10 13:44:53.586067 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:53.586052 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb09abf0-11a7-48a7-9138-015aaa58002b" containerName="payload-processing" Sep 10 13:44:53.586067 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:53.586068 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb09abf0-11a7-48a7-9138-015aaa58002b" containerName="payload-processing" Sep 10 13:44:53.586204 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:53.586082 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebacc7db-cacd-4d21-a123-d087c763037c" containerName="payload-pre-processing" Sep 10 13:44:53.586204 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:53.586087 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebacc7db-cacd-4d21-a123-d087c763037c" containerName="payload-pre-processing" Sep 10 13:44:53.586204 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:53.586164 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb09abf0-11a7-48a7-9138-015aaa58002b" containerName="payload-processing" Sep 10 13:44:53.586204 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:53.586173 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebacc7db-cacd-4d21-a123-d087c763037c" containerName="payload-pre-processing" Sep 10 13:44:53.589268 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:53.589251 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-a2d945-c7677fc7c-b8jdd" Sep 10 13:44:53.592300 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:53.592276 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-a2d945\"" Sep 10 13:44:53.592422 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:53.592277 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-a2d945-dockercfg-ddl5b\"" Sep 10 13:44:53.601809 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:53.601777 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a2d945-c7677fc7c-b8jdd"] Sep 10 13:44:53.636915 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:53.636872 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns5zw\" (UniqueName: \"kubernetes.io/projected/4462dd2c-1648-4139-b60c-7ee68956bfd3-kube-api-access-ns5zw\") pod \"payload-processing-e2e-api-a-a2d945-c7677fc7c-b8jdd\" (UID: \"4462dd2c-1648-4139-b60c-7ee68956bfd3\") " pod="openshift-ingress/payload-processing-e2e-api-a-a2d945-c7677fc7c-b8jdd" Sep 10 13:44:53.637107 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:53.636962 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4462dd2c-1648-4139-b60c-7ee68956bfd3-plugins-config-volume\") pod \"payload-processing-e2e-api-a-a2d945-c7677fc7c-b8jdd\" (UID: \"4462dd2c-1648-4139-b60c-7ee68956bfd3\") " pod="openshift-ingress/payload-processing-e2e-api-a-a2d945-c7677fc7c-b8jdd" Sep 10 13:44:53.651836 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:53.651800 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a2d945-796fb9554b-24rc4"] Sep 10 13:44:53.655347 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:53.655330 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a2d945-796fb9554b-24rc4" Sep 10 13:44:53.666079 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:53.666050 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a2d945-796fb9554b-24rc4"] Sep 10 13:44:53.738533 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:53.738477 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns5zw\" (UniqueName: \"kubernetes.io/projected/4462dd2c-1648-4139-b60c-7ee68956bfd3-kube-api-access-ns5zw\") pod \"payload-processing-e2e-api-a-a2d945-c7677fc7c-b8jdd\" (UID: \"4462dd2c-1648-4139-b60c-7ee68956bfd3\") " pod="openshift-ingress/payload-processing-e2e-api-a-a2d945-c7677fc7c-b8jdd" Sep 10 13:44:53.738759 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:53.738568 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hz5qx\" (UniqueName: \"kubernetes.io/projected/8f58519d-aee2-4789-b75b-e55117a9ab07-kube-api-access-hz5qx\") pod \"payload-pre-processing-e2e-api-a-a2d945-796fb9554b-24rc4\" (UID: \"8f58519d-aee2-4789-b75b-e55117a9ab07\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a2d945-796fb9554b-24rc4" Sep 10 13:44:53.738759 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:53.738603 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4462dd2c-1648-4139-b60c-7ee68956bfd3-plugins-config-volume\") pod \"payload-processing-e2e-api-a-a2d945-c7677fc7c-b8jdd\" (UID: \"4462dd2c-1648-4139-b60c-7ee68956bfd3\") " pod="openshift-ingress/payload-processing-e2e-api-a-a2d945-c7677fc7c-b8jdd" Sep 10 13:44:53.738759 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:53.738662 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f58519d-aee2-4789-b75b-e55117a9ab07-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-a2d945-796fb9554b-24rc4\" (UID: \"8f58519d-aee2-4789-b75b-e55117a9ab07\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a2d945-796fb9554b-24rc4" Sep 10 13:44:53.740074 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:53.740042 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4462dd2c-1648-4139-b60c-7ee68956bfd3-plugins-config-volume\") pod \"payload-processing-e2e-api-a-a2d945-c7677fc7c-b8jdd\" (UID: \"4462dd2c-1648-4139-b60c-7ee68956bfd3\") " pod="openshift-ingress/payload-processing-e2e-api-a-a2d945-c7677fc7c-b8jdd" Sep 10 13:44:53.747618 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:53.747592 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns5zw\" (UniqueName: \"kubernetes.io/projected/4462dd2c-1648-4139-b60c-7ee68956bfd3-kube-api-access-ns5zw\") pod \"payload-processing-e2e-api-a-a2d945-c7677fc7c-b8jdd\" (UID: \"4462dd2c-1648-4139-b60c-7ee68956bfd3\") " pod="openshift-ingress/payload-processing-e2e-api-a-a2d945-c7677fc7c-b8jdd" Sep 10 13:44:53.839996 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:53.839896 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hz5qx\" (UniqueName: \"kubernetes.io/projected/8f58519d-aee2-4789-b75b-e55117a9ab07-kube-api-access-hz5qx\") pod \"payload-pre-processing-e2e-api-a-a2d945-796fb9554b-24rc4\" (UID: \"8f58519d-aee2-4789-b75b-e55117a9ab07\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a2d945-796fb9554b-24rc4" Sep 10 13:44:53.839996 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:53.839961 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f58519d-aee2-4789-b75b-e55117a9ab07-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-a2d945-796fb9554b-24rc4\" (UID: \"8f58519d-aee2-4789-b75b-e55117a9ab07\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a2d945-796fb9554b-24rc4" Sep 10 13:44:53.840705 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:53.840678 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f58519d-aee2-4789-b75b-e55117a9ab07-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-a2d945-796fb9554b-24rc4\" (UID: \"8f58519d-aee2-4789-b75b-e55117a9ab07\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a2d945-796fb9554b-24rc4" Sep 10 13:44:53.849175 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:53.849149 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hz5qx\" (UniqueName: \"kubernetes.io/projected/8f58519d-aee2-4789-b75b-e55117a9ab07-kube-api-access-hz5qx\") pod \"payload-pre-processing-e2e-api-a-a2d945-796fb9554b-24rc4\" (UID: \"8f58519d-aee2-4789-b75b-e55117a9ab07\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a2d945-796fb9554b-24rc4" Sep 10 13:44:53.901324 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:53.901282 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-a2d945-c7677fc7c-b8jdd" Sep 10 13:44:53.966470 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:53.966157 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a2d945-796fb9554b-24rc4" Sep 10 13:44:54.181281 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:54.181248 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a2d945-c7677fc7c-b8jdd"] Sep 10 13:44:54.183414 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:44:54.183372 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4462dd2c_1648_4139_b60c_7ee68956bfd3.slice/crio-6a550f3a69ead140fe42add57586621723dae0d877ff519be2efd40c8f528d96 WatchSource:0}: Error finding container 6a550f3a69ead140fe42add57586621723dae0d877ff519be2efd40c8f528d96: Status 404 returned error can't find the container with id 6a550f3a69ead140fe42add57586621723dae0d877ff519be2efd40c8f528d96 Sep 10 13:44:54.249751 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:54.249722 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a2d945-796fb9554b-24rc4"] Sep 10 13:44:54.252147 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:44:54.252117 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f58519d_aee2_4789_b75b_e55117a9ab07.slice/crio-59a45ab5502b00307a1408a89c6b764160c3e874d0d7ff45db5a085a051178ef WatchSource:0}: Error finding container 59a45ab5502b00307a1408a89c6b764160c3e874d0d7ff45db5a085a051178ef: Status 404 returned error can't find the container with id 59a45ab5502b00307a1408a89c6b764160c3e874d0d7ff45db5a085a051178ef Sep 10 13:44:55.055669 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:55.055619 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-a2d945-c7677fc7c-b8jdd" event={"ID":"4462dd2c-1648-4139-b60c-7ee68956bfd3","Type":"ContainerStarted","Data":"0c59d498f8b6be02d3585a14118f48105fb8892de42390d5697e5e616955155d"} Sep 10 13:44:55.056117 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:55.055688 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-a2d945-c7677fc7c-b8jdd" event={"ID":"4462dd2c-1648-4139-b60c-7ee68956bfd3","Type":"ContainerStarted","Data":"6a550f3a69ead140fe42add57586621723dae0d877ff519be2efd40c8f528d96"} Sep 10 13:44:55.056117 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:55.055717 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-a2d945-c7677fc7c-b8jdd" Sep 10 13:44:55.057311 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:55.057285 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a2d945-796fb9554b-24rc4" event={"ID":"8f58519d-aee2-4789-b75b-e55117a9ab07","Type":"ContainerStarted","Data":"499b5d2794b343bfaf72afa56e0d8eaf65d30fcd92a3eb44f28ed8475204ba97"} Sep 10 13:44:55.057437 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:55.057334 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a2d945-796fb9554b-24rc4" event={"ID":"8f58519d-aee2-4789-b75b-e55117a9ab07","Type":"ContainerStarted","Data":"59a45ab5502b00307a1408a89c6b764160c3e874d0d7ff45db5a085a051178ef"} Sep 10 13:44:55.057437 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:55.057374 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a2d945-796fb9554b-24rc4" Sep 10 13:44:55.076571 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:55.076516 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-a2d945-c7677fc7c-b8jdd" podStartSLOduration=2.076500846 podStartE2EDuration="2.076500846s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:44:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:44:55.073915917 +0000 UTC m=+1604.968633336" watchObservedRunningTime="2026-09-10 13:44:55.076500846 +0000 UTC m=+1604.971218264" Sep 10 13:44:55.093270 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:44:55.093222 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a2d945-796fb9554b-24rc4" podStartSLOduration=2.093206273 podStartE2EDuration="2.093206273s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:44:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:44:55.090818385 +0000 UTC m=+1604.985535804" watchObservedRunningTime="2026-09-10 13:44:55.093206273 +0000 UTC m=+1604.987923691" Sep 10 13:45:00.139161 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:00.139119 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-ba174e-29817465-fdl7c"] Sep 10 13:45:00.143326 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:00.143285 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-ba174e-29817465-fdl7c" Sep 10 13:45:00.146019 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:00.145989 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-5d75ac-29817465-97xwj"] Sep 10 13:45:00.146295 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:00.146277 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pffwn\"" Sep 10 13:45:00.149784 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:00.149759 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-5d75ac-29817465-97xwj" Sep 10 13:45:00.153509 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:00.153483 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-ba174e-29817465-fdl7c"] Sep 10 13:45:00.157019 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:00.156975 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-ba174e-29817465-fdl7c" Sep 10 13:45:00.162335 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:00.162316 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-5d75ac-29817465-97xwj"] Sep 10 13:45:00.166868 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:00.166203 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a2d945-29817465-2dbjz"] Sep 10 13:45:00.169220 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:00.169198 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-5d75ac-29817465-97xwj" Sep 10 13:45:00.170863 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:00.170842 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a2d945-29817465-2dbjz" Sep 10 13:45:00.178684 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:00.178658 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a2d945-29817465-2dbjz"] Sep 10 13:45:00.224014 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:00.223989 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a2d945-29817465-2dbjz" Sep 10 13:45:00.470913 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:00.470886 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-ba174e-29817465-fdl7c"] Sep 10 13:45:00.472317 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:45:00.472293 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4e27dd4_c5c8_4bea_a236_07fb72a40da2.slice/crio-ca6a8df3c05c0215135cef50f970f89d28c0c6b6e5d2ff164e62c3dfd0789ef8 WatchSource:0}: Error finding container ca6a8df3c05c0215135cef50f970f89d28c0c6b6e5d2ff164e62c3dfd0789ef8: Status 404 returned error can't find the container with id ca6a8df3c05c0215135cef50f970f89d28c0c6b6e5d2ff164e62c3dfd0789ef8 Sep 10 13:45:00.482665 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:00.482643 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-5d75ac-29817465-97xwj"] Sep 10 13:45:00.486042 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:45:00.486014 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8160c0d1_974d_42cb_9802_49b78cf5ec43.slice/crio-a590e2c9e5e1699640b6c9d359da966e847ac0a700413ec98a1d5b0e655f2f5f WatchSource:0}: Error finding container a590e2c9e5e1699640b6c9d359da966e847ac0a700413ec98a1d5b0e655f2f5f: Status 404 returned error can't find the container with id a590e2c9e5e1699640b6c9d359da966e847ac0a700413ec98a1d5b0e655f2f5f Sep 10 13:45:00.543355 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:00.543330 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a2d945-29817465-2dbjz"] Sep 10 13:45:00.545037 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:45:00.545009 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod709f5a1a_4ce6_4b54_9b9a_b273a51a3eae.slice/crio-6c3db6b0dde50623a851f77d466ea4e6a07afe4295fbb8670b61bba6d25a560c WatchSource:0}: Error finding container 6c3db6b0dde50623a851f77d466ea4e6a07afe4295fbb8670b61bba6d25a560c: Status 404 returned error can't find the container with id 6c3db6b0dde50623a851f77d466ea4e6a07afe4295fbb8670b61bba6d25a560c Sep 10 13:45:01.095673 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:01.095564 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-ba174e-29817465-fdl7c" event={"ID":"e4e27dd4-c5c8-4bea-a236-07fb72a40da2","Type":"ContainerStarted","Data":"46f1ee33089edd877f2e1bddaf5227123f666cfbced0581d193e3bade84afc09"} Sep 10 13:45:01.095673 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:01.095626 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-ba174e-29817465-fdl7c" event={"ID":"e4e27dd4-c5c8-4bea-a236-07fb72a40da2","Type":"ContainerStarted","Data":"ca6a8df3c05c0215135cef50f970f89d28c0c6b6e5d2ff164e62c3dfd0789ef8"} Sep 10 13:45:01.097308 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:01.097278 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-5d75ac-29817465-97xwj" event={"ID":"8160c0d1-974d-42cb-9802-49b78cf5ec43","Type":"ContainerStarted","Data":"b72d20a4e40e5d1dc4964fdb3000f3d4dda6b13df0d2c7226ba09de4df400d6a"} Sep 10 13:45:01.097448 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:01.097332 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-5d75ac-29817465-97xwj" event={"ID":"8160c0d1-974d-42cb-9802-49b78cf5ec43","Type":"ContainerStarted","Data":"a590e2c9e5e1699640b6c9d359da966e847ac0a700413ec98a1d5b0e655f2f5f"} Sep 10 13:45:01.098816 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:01.098795 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a2d945-29817465-2dbjz" event={"ID":"709f5a1a-4ce6-4b54-9b9a-b273a51a3eae","Type":"ContainerStarted","Data":"5824ebf61a42f9914c12a825b375d5bdeabff5e3adca2619c4defd91347e671b"} Sep 10 13:45:01.098910 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:01.098833 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a2d945-29817465-2dbjz" event={"ID":"709f5a1a-4ce6-4b54-9b9a-b273a51a3eae","Type":"ContainerStarted","Data":"6c3db6b0dde50623a851f77d466ea4e6a07afe4295fbb8670b61bba6d25a560c"} Sep 10 13:45:01.118153 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:01.118096 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-ba174e-29817465-fdl7c" podStartSLOduration=1.118078988 podStartE2EDuration="1.118078988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:45:01.114885639 +0000 UTC m=+1611.009603058" watchObservedRunningTime="2026-09-10 13:45:01.118078988 +0000 UTC m=+1611.012796407" Sep 10 13:45:01.135976 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:01.135927 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-5d75ac-29817465-97xwj" podStartSLOduration=1.135909099 podStartE2EDuration="1.135909099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:45:01.134581107 +0000 UTC m=+1611.029298552" watchObservedRunningTime="2026-09-10 13:45:01.135909099 +0000 UTC m=+1611.030626517" Sep 10 13:45:01.155936 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:01.155880 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a2d945-29817465-2dbjz" podStartSLOduration=1.155862353 podStartE2EDuration="1.155862353s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:45:01.153602391 +0000 UTC m=+1611.048319812" watchObservedRunningTime="2026-09-10 13:45:01.155862353 +0000 UTC m=+1611.050579771" Sep 10 13:45:02.106995 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:02.106951 2560 generic.go:356] "Generic (PLEG): container finished" podID="e4e27dd4-c5c8-4bea-a236-07fb72a40da2" containerID="46f1ee33089edd877f2e1bddaf5227123f666cfbced0581d193e3bade84afc09" exitCode=0 Sep 10 13:45:02.107158 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:02.107038 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-ba174e-29817465-fdl7c" event={"ID":"e4e27dd4-c5c8-4bea-a236-07fb72a40da2","Type":"ContainerDied","Data":"46f1ee33089edd877f2e1bddaf5227123f666cfbced0581d193e3bade84afc09"} Sep 10 13:45:02.115957 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:02.115923 2560 generic.go:356] "Generic (PLEG): container finished" podID="8160c0d1-974d-42cb-9802-49b78cf5ec43" containerID="b72d20a4e40e5d1dc4964fdb3000f3d4dda6b13df0d2c7226ba09de4df400d6a" exitCode=0 Sep 10 13:45:02.116118 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:02.116013 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-5d75ac-29817465-97xwj" event={"ID":"8160c0d1-974d-42cb-9802-49b78cf5ec43","Type":"ContainerDied","Data":"b72d20a4e40e5d1dc4964fdb3000f3d4dda6b13df0d2c7226ba09de4df400d6a"} Sep 10 13:45:02.117742 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:02.117720 2560 generic.go:356] "Generic (PLEG): container finished" podID="709f5a1a-4ce6-4b54-9b9a-b273a51a3eae" containerID="5824ebf61a42f9914c12a825b375d5bdeabff5e3adca2619c4defd91347e671b" exitCode=0 Sep 10 13:45:02.117871 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:02.117743 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a2d945-29817465-2dbjz" event={"ID":"709f5a1a-4ce6-4b54-9b9a-b273a51a3eae","Type":"ContainerDied","Data":"5824ebf61a42f9914c12a825b375d5bdeabff5e3adca2619c4defd91347e671b"} Sep 10 13:45:03.287552 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:03.287524 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-5d75ac-29817465-97xwj" Sep 10 13:45:03.334947 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:03.334922 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a2d945-29817465-2dbjz" Sep 10 13:45:03.338742 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:03.338717 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-ba174e-29817465-fdl7c" Sep 10 13:45:04.132277 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:04.132240 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-5d75ac-29817465-97xwj" Sep 10 13:45:04.132491 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:04.132273 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-5d75ac-29817465-97xwj" event={"ID":"8160c0d1-974d-42cb-9802-49b78cf5ec43","Type":"ContainerDied","Data":"a590e2c9e5e1699640b6c9d359da966e847ac0a700413ec98a1d5b0e655f2f5f"} Sep 10 13:45:04.132491 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:04.132332 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a590e2c9e5e1699640b6c9d359da966e847ac0a700413ec98a1d5b0e655f2f5f" Sep 10 13:45:04.133706 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:04.133683 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a2d945-29817465-2dbjz" Sep 10 13:45:04.133839 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:04.133733 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a2d945-29817465-2dbjz" event={"ID":"709f5a1a-4ce6-4b54-9b9a-b273a51a3eae","Type":"ContainerDied","Data":"6c3db6b0dde50623a851f77d466ea4e6a07afe4295fbb8670b61bba6d25a560c"} Sep 10 13:45:04.133839 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:04.133782 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6c3db6b0dde50623a851f77d466ea4e6a07afe4295fbb8670b61bba6d25a560c" Sep 10 13:45:04.135310 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:04.135283 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-ba174e-29817465-fdl7c" event={"ID":"e4e27dd4-c5c8-4bea-a236-07fb72a40da2","Type":"ContainerDied","Data":"ca6a8df3c05c0215135cef50f970f89d28c0c6b6e5d2ff164e62c3dfd0789ef8"} Sep 10 13:45:04.135425 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:04.135326 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca6a8df3c05c0215135cef50f970f89d28c0c6b6e5d2ff164e62c3dfd0789ef8" Sep 10 13:45:04.135425 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:04.135337 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-ba174e-29817465-fdl7c" Sep 10 13:45:06.065825 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:06.065790 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a2d945-796fb9554b-24rc4" Sep 10 13:45:06.066319 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:06.066018 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-a2d945-c7677fc7c-b8jdd" Sep 10 13:45:10.757726 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:10.757695 2560 scope.go:117] "RemoveContainer" containerID="0bf7f4c45b0af8efa1d0c06f46c447396113392fe69a8a601d4f15c4cdaad1ee" Sep 10 13:45:10.766793 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:10.766771 2560 scope.go:117] "RemoveContainer" containerID="630a6db4ef3639ae6483258a99d1f643861c7b51532a005ca552a168c5493deb" Sep 10 13:45:16.273500 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:16.273465 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:45:16.490636 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:16.490601 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:45:16.721390 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:16.721334 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:45:17.175827 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:17.175792 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:45:17.525502 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:17.525397 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:45:17.623031 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:17.623000 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:45:20.994185 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:20.994145 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:45:21.124501 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:21.124462 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:45:22.324233 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:22.324190 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:45:22.425654 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:22.425622 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:45:22.701960 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:22.701925 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:45:22.879251 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:22.879210 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:45:23.012333 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:23.012242 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-02a403-7575c48fc-ztgqf"] Sep 10 13:45:23.012817 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:23.012786 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="709f5a1a-4ce6-4b54-9b9a-b273a51a3eae" containerName="cleanup" Sep 10 13:45:23.012817 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:23.012808 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="709f5a1a-4ce6-4b54-9b9a-b273a51a3eae" containerName="cleanup" Sep 10 13:45:23.012969 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:23.012826 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4e27dd4-c5c8-4bea-a236-07fb72a40da2" containerName="cleanup" Sep 10 13:45:23.012969 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:23.012834 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4e27dd4-c5c8-4bea-a236-07fb72a40da2" containerName="cleanup" Sep 10 13:45:23.012969 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:23.012876 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8160c0d1-974d-42cb-9802-49b78cf5ec43" containerName="cleanup" Sep 10 13:45:23.012969 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:23.012885 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8160c0d1-974d-42cb-9802-49b78cf5ec43" containerName="cleanup" Sep 10 13:45:23.013176 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:23.013002 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="709f5a1a-4ce6-4b54-9b9a-b273a51a3eae" containerName="cleanup" Sep 10 13:45:23.013176 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:23.013017 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4e27dd4-c5c8-4bea-a236-07fb72a40da2" containerName="cleanup" Sep 10 13:45:23.013176 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:23.013025 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8160c0d1-974d-42cb-9802-49b78cf5ec43" containerName="cleanup" Sep 10 13:45:23.016486 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:23.016463 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-02a403-7575c48fc-ztgqf" Sep 10 13:45:23.018707 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:23.018683 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-02a403\"" Sep 10 13:45:23.018826 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:23.018758 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-02a403-dockercfg-wkv97\"" Sep 10 13:45:23.023329 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:23.023125 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad82f10d-fa27-43d2-9e46-1d70e172357e-plugins-config-volume\") pod \"payload-processing-e2e-api-b-02a403-7575c48fc-ztgqf\" (UID: \"ad82f10d-fa27-43d2-9e46-1d70e172357e\") " pod="openshift-ingress/payload-processing-e2e-api-b-02a403-7575c48fc-ztgqf" Sep 10 13:45:23.023506 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:23.023427 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsqqz\" (UniqueName: \"kubernetes.io/projected/ad82f10d-fa27-43d2-9e46-1d70e172357e-kube-api-access-zsqqz\") pod \"payload-processing-e2e-api-b-02a403-7575c48fc-ztgqf\" (UID: \"ad82f10d-fa27-43d2-9e46-1d70e172357e\") " pod="openshift-ingress/payload-processing-e2e-api-b-02a403-7575c48fc-ztgqf" Sep 10 13:45:23.027903 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:23.027876 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-02a403-7575c48fc-ztgqf"] Sep 10 13:45:23.069334 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:23.069306 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-02a403-5569f8fc5d-mwn4z"] Sep 10 13:45:23.073117 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:23.073097 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-02a403-5569f8fc5d-mwn4z" Sep 10 13:45:23.084543 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:23.084519 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-02a403-5569f8fc5d-mwn4z"] Sep 10 13:45:23.107944 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:23.107900 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:45:23.124560 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:23.124508 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87c726c0-75d8-4960-b5e0-6be72150b117-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-02a403-5569f8fc5d-mwn4z\" (UID: \"87c726c0-75d8-4960-b5e0-6be72150b117\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-02a403-5569f8fc5d-mwn4z" Sep 10 13:45:23.124757 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:23.124576 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsqqz\" (UniqueName: \"kubernetes.io/projected/ad82f10d-fa27-43d2-9e46-1d70e172357e-kube-api-access-zsqqz\") pod \"payload-processing-e2e-api-b-02a403-7575c48fc-ztgqf\" (UID: \"ad82f10d-fa27-43d2-9e46-1d70e172357e\") " pod="openshift-ingress/payload-processing-e2e-api-b-02a403-7575c48fc-ztgqf" Sep 10 13:45:23.124757 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:23.124676 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad82f10d-fa27-43d2-9e46-1d70e172357e-plugins-config-volume\") pod \"payload-processing-e2e-api-b-02a403-7575c48fc-ztgqf\" (UID: \"ad82f10d-fa27-43d2-9e46-1d70e172357e\") " pod="openshift-ingress/payload-processing-e2e-api-b-02a403-7575c48fc-ztgqf" Sep 10 13:45:23.124874 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:23.124761 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cm5tf\" (UniqueName: \"kubernetes.io/projected/87c726c0-75d8-4960-b5e0-6be72150b117-kube-api-access-cm5tf\") pod \"payload-pre-processing-e2e-api-b-02a403-5569f8fc5d-mwn4z\" (UID: \"87c726c0-75d8-4960-b5e0-6be72150b117\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-02a403-5569f8fc5d-mwn4z" Sep 10 13:45:23.125440 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:23.125394 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad82f10d-fa27-43d2-9e46-1d70e172357e-plugins-config-volume\") pod \"payload-processing-e2e-api-b-02a403-7575c48fc-ztgqf\" (UID: \"ad82f10d-fa27-43d2-9e46-1d70e172357e\") " pod="openshift-ingress/payload-processing-e2e-api-b-02a403-7575c48fc-ztgqf" Sep 10 13:45:23.138518 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:23.138130 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsqqz\" (UniqueName: \"kubernetes.io/projected/ad82f10d-fa27-43d2-9e46-1d70e172357e-kube-api-access-zsqqz\") pod \"payload-processing-e2e-api-b-02a403-7575c48fc-ztgqf\" (UID: \"ad82f10d-fa27-43d2-9e46-1d70e172357e\") " pod="openshift-ingress/payload-processing-e2e-api-b-02a403-7575c48fc-ztgqf" Sep 10 13:45:23.225278 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:23.225239 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cm5tf\" (UniqueName: \"kubernetes.io/projected/87c726c0-75d8-4960-b5e0-6be72150b117-kube-api-access-cm5tf\") pod \"payload-pre-processing-e2e-api-b-02a403-5569f8fc5d-mwn4z\" (UID: \"87c726c0-75d8-4960-b5e0-6be72150b117\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-02a403-5569f8fc5d-mwn4z" Sep 10 13:45:23.225278 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:23.225283 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87c726c0-75d8-4960-b5e0-6be72150b117-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-02a403-5569f8fc5d-mwn4z\" (UID: \"87c726c0-75d8-4960-b5e0-6be72150b117\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-02a403-5569f8fc5d-mwn4z" Sep 10 13:45:23.225970 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:23.225953 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87c726c0-75d8-4960-b5e0-6be72150b117-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-02a403-5569f8fc5d-mwn4z\" (UID: \"87c726c0-75d8-4960-b5e0-6be72150b117\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-02a403-5569f8fc5d-mwn4z" Sep 10 13:45:23.236872 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:23.236842 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cm5tf\" (UniqueName: \"kubernetes.io/projected/87c726c0-75d8-4960-b5e0-6be72150b117-kube-api-access-cm5tf\") pod \"payload-pre-processing-e2e-api-b-02a403-5569f8fc5d-mwn4z\" (UID: \"87c726c0-75d8-4960-b5e0-6be72150b117\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-02a403-5569f8fc5d-mwn4z" Sep 10 13:45:23.327953 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:23.327858 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-02a403-7575c48fc-ztgqf" Sep 10 13:45:23.383736 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:23.383702 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-02a403-5569f8fc5d-mwn4z" Sep 10 13:45:23.605784 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:23.605754 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-02a403-7575c48fc-ztgqf"] Sep 10 13:45:23.606649 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:45:23.606623 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad82f10d_fa27_43d2_9e46_1d70e172357e.slice/crio-041429cdbbd045b42e88aa9ddccd1be99e04cf4fc1660f8c9afc5a55b4f33856 WatchSource:0}: Error finding container 041429cdbbd045b42e88aa9ddccd1be99e04cf4fc1660f8c9afc5a55b4f33856: Status 404 returned error can't find the container with id 041429cdbbd045b42e88aa9ddccd1be99e04cf4fc1660f8c9afc5a55b4f33856 Sep 10 13:45:23.674177 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:23.674146 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-02a403-5569f8fc5d-mwn4z"] Sep 10 13:45:23.675029 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:45:23.675007 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87c726c0_75d8_4960_b5e0_6be72150b117.slice/crio-9f1446066af5b98f8e9b13b2cdce4f2bf0378b750af74f92ae7f1cc92f2e5163 WatchSource:0}: Error finding container 9f1446066af5b98f8e9b13b2cdce4f2bf0378b750af74f92ae7f1cc92f2e5163: Status 404 returned error can't find the container with id 9f1446066af5b98f8e9b13b2cdce4f2bf0378b750af74f92ae7f1cc92f2e5163 Sep 10 13:45:24.265562 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:24.265516 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-02a403-7575c48fc-ztgqf" event={"ID":"ad82f10d-fa27-43d2-9e46-1d70e172357e","Type":"ContainerStarted","Data":"db10cd17cdb217076441c4909230b204ad08c761b116067a7128af3e887e6b63"} Sep 10 13:45:24.265762 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:24.265689 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-02a403-7575c48fc-ztgqf" Sep 10 13:45:24.265762 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:24.265731 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-02a403-7575c48fc-ztgqf" event={"ID":"ad82f10d-fa27-43d2-9e46-1d70e172357e","Type":"ContainerStarted","Data":"041429cdbbd045b42e88aa9ddccd1be99e04cf4fc1660f8c9afc5a55b4f33856"} Sep 10 13:45:24.267328 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:24.267296 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-02a403-5569f8fc5d-mwn4z" event={"ID":"87c726c0-75d8-4960-b5e0-6be72150b117","Type":"ContainerStarted","Data":"9d1730d90fb21ece8407b7ad3903ca6e3b7d9634f3f777f480045ec3762f7f90"} Sep 10 13:45:24.267470 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:24.267337 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-02a403-5569f8fc5d-mwn4z" event={"ID":"87c726c0-75d8-4960-b5e0-6be72150b117","Type":"ContainerStarted","Data":"9f1446066af5b98f8e9b13b2cdce4f2bf0378b750af74f92ae7f1cc92f2e5163"} Sep 10 13:45:24.267470 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:24.267453 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-02a403-5569f8fc5d-mwn4z" Sep 10 13:45:24.287706 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:24.287648 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-02a403-7575c48fc-ztgqf" podStartSLOduration=2.2876269049999998 podStartE2EDuration="2.287626905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:45:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:45:24.284015321 +0000 UTC m=+1634.178732741" watchObservedRunningTime="2026-09-10 13:45:24.287626905 +0000 UTC m=+1634.182344326" Sep 10 13:45:24.303368 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:24.303311 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-02a403-5569f8fc5d-mwn4z" podStartSLOduration=1.303292698 podStartE2EDuration="1.303292698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:45:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:45:24.300170931 +0000 UTC m=+1634.194888364" watchObservedRunningTime="2026-09-10 13:45:24.303292698 +0000 UTC m=+1634.198010116" Sep 10 13:45:35.276600 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:35.276567 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-02a403-7575c48fc-ztgqf" Sep 10 13:45:35.276993 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:35.276750 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-02a403-5569f8fc5d-mwn4z" Sep 10 13:45:41.683276 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:41.683243 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:45:41.784963 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:41.784933 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:45:46.017496 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:46.017459 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:45:49.100452 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:45:49.100415 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:46:00.112884 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:00.112849 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:46:03.797741 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:03.797698 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:46:04.817102 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:04.817063 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a2d945-29817465-2dbjz"] Sep 10 13:46:04.820218 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:04.820188 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-a2d945-29817465-2dbjz"] Sep 10 13:46:04.841703 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:04.841666 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:46:04.849702 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:04.849672 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a2d945-c7677fc7c-b8jdd"] Sep 10 13:46:04.849924 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:04.849897 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-a2d945-c7677fc7c-b8jdd" podUID="4462dd2c-1648-4139-b60c-7ee68956bfd3" containerName="payload-processing" containerID="cri-o://0c59d498f8b6be02d3585a14118f48105fb8892de42390d5697e5e616955155d" gracePeriod=30 Sep 10 13:46:04.859468 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:04.859433 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a2d945-796fb9554b-24rc4"] Sep 10 13:46:04.860035 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:04.859771 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a2d945-796fb9554b-24rc4" podUID="8f58519d-aee2-4789-b75b-e55117a9ab07" containerName="payload-pre-processing" containerID="cri-o://499b5d2794b343bfaf72afa56e0d8eaf65d30fcd92a3eb44f28ed8475204ba97" gracePeriod=30 Sep 10 13:46:04.942031 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:04.941984 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:46:05.039466 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:05.039439 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:46:05.117281 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:05.116992 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:46:05.133645 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:05.133589 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-a2d945-c7677fc7c-b8jdd" Sep 10 13:46:05.166243 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:05.166212 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a2d945-796fb9554b-24rc4" Sep 10 13:46:05.315193 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:05.315157 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4462dd2c-1648-4139-b60c-7ee68956bfd3-plugins-config-volume\") pod \"4462dd2c-1648-4139-b60c-7ee68956bfd3\" (UID: \"4462dd2c-1648-4139-b60c-7ee68956bfd3\") " Sep 10 13:46:05.315396 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:05.315200 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ns5zw\" (UniqueName: \"kubernetes.io/projected/4462dd2c-1648-4139-b60c-7ee68956bfd3-kube-api-access-ns5zw\") pod \"4462dd2c-1648-4139-b60c-7ee68956bfd3\" (UID: \"4462dd2c-1648-4139-b60c-7ee68956bfd3\") " Sep 10 13:46:05.315396 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:05.315255 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f58519d-aee2-4789-b75b-e55117a9ab07-plugins-config-volume\") pod \"8f58519d-aee2-4789-b75b-e55117a9ab07\" (UID: \"8f58519d-aee2-4789-b75b-e55117a9ab07\") " Sep 10 13:46:05.315396 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:05.315347 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hz5qx\" (UniqueName: \"kubernetes.io/projected/8f58519d-aee2-4789-b75b-e55117a9ab07-kube-api-access-hz5qx\") pod \"8f58519d-aee2-4789-b75b-e55117a9ab07\" (UID: \"8f58519d-aee2-4789-b75b-e55117a9ab07\") " Sep 10 13:46:05.315611 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:05.315577 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4462dd2c-1648-4139-b60c-7ee68956bfd3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4462dd2c-1648-4139-b60c-7ee68956bfd3" (UID: "4462dd2c-1648-4139-b60c-7ee68956bfd3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:46:05.315693 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:05.315670 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f58519d-aee2-4789-b75b-e55117a9ab07-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8f58519d-aee2-4789-b75b-e55117a9ab07" (UID: "8f58519d-aee2-4789-b75b-e55117a9ab07"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:46:05.315741 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:05.315689 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4462dd2c-1648-4139-b60c-7ee68956bfd3-plugins-config-volume\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:46:05.317410 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:05.317383 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4462dd2c-1648-4139-b60c-7ee68956bfd3-kube-api-access-ns5zw" (OuterVolumeSpecName: "kube-api-access-ns5zw") pod "4462dd2c-1648-4139-b60c-7ee68956bfd3" (UID: "4462dd2c-1648-4139-b60c-7ee68956bfd3"). InnerVolumeSpecName "kube-api-access-ns5zw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:46:05.317475 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:05.317442 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f58519d-aee2-4789-b75b-e55117a9ab07-kube-api-access-hz5qx" (OuterVolumeSpecName: "kube-api-access-hz5qx") pod "8f58519d-aee2-4789-b75b-e55117a9ab07" (UID: "8f58519d-aee2-4789-b75b-e55117a9ab07"). InnerVolumeSpecName "kube-api-access-hz5qx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:46:05.418239 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:05.418190 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hz5qx\" (UniqueName: \"kubernetes.io/projected/8f58519d-aee2-4789-b75b-e55117a9ab07-kube-api-access-hz5qx\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:46:05.418239 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:05.418241 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ns5zw\" (UniqueName: \"kubernetes.io/projected/4462dd2c-1648-4139-b60c-7ee68956bfd3-kube-api-access-ns5zw\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:46:05.418565 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:05.418257 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f58519d-aee2-4789-b75b-e55117a9ab07-plugins-config-volume\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:46:05.439164 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:05.439130 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:46:05.525706 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:05.525671 2560 generic.go:356] "Generic (PLEG): container finished" podID="8f58519d-aee2-4789-b75b-e55117a9ab07" containerID="499b5d2794b343bfaf72afa56e0d8eaf65d30fcd92a3eb44f28ed8475204ba97" exitCode=0 Sep 10 13:46:05.525899 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:05.525725 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a2d945-796fb9554b-24rc4" Sep 10 13:46:05.525899 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:05.525753 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a2d945-796fb9554b-24rc4" event={"ID":"8f58519d-aee2-4789-b75b-e55117a9ab07","Type":"ContainerDied","Data":"499b5d2794b343bfaf72afa56e0d8eaf65d30fcd92a3eb44f28ed8475204ba97"} Sep 10 13:46:05.525899 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:05.525805 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a2d945-796fb9554b-24rc4" event={"ID":"8f58519d-aee2-4789-b75b-e55117a9ab07","Type":"ContainerDied","Data":"59a45ab5502b00307a1408a89c6b764160c3e874d0d7ff45db5a085a051178ef"} Sep 10 13:46:05.525899 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:05.525837 2560 scope.go:117] "RemoveContainer" containerID="499b5d2794b343bfaf72afa56e0d8eaf65d30fcd92a3eb44f28ed8475204ba97" Sep 10 13:46:05.527242 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:05.527218 2560 generic.go:356] "Generic (PLEG): container finished" podID="4462dd2c-1648-4139-b60c-7ee68956bfd3" containerID="0c59d498f8b6be02d3585a14118f48105fb8892de42390d5697e5e616955155d" exitCode=0 Sep 10 13:46:05.527365 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:05.527258 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-a2d945-c7677fc7c-b8jdd" Sep 10 13:46:05.527365 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:05.527324 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-a2d945-c7677fc7c-b8jdd" event={"ID":"4462dd2c-1648-4139-b60c-7ee68956bfd3","Type":"ContainerDied","Data":"0c59d498f8b6be02d3585a14118f48105fb8892de42390d5697e5e616955155d"} Sep 10 13:46:05.527480 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:05.527368 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-a2d945-c7677fc7c-b8jdd" event={"ID":"4462dd2c-1648-4139-b60c-7ee68956bfd3","Type":"ContainerDied","Data":"6a550f3a69ead140fe42add57586621723dae0d877ff519be2efd40c8f528d96"} Sep 10 13:46:05.536077 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:05.535965 2560 scope.go:117] "RemoveContainer" containerID="499b5d2794b343bfaf72afa56e0d8eaf65d30fcd92a3eb44f28ed8475204ba97" Sep 10 13:46:05.536395 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:46:05.536366 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"499b5d2794b343bfaf72afa56e0d8eaf65d30fcd92a3eb44f28ed8475204ba97\": container with ID starting with 499b5d2794b343bfaf72afa56e0d8eaf65d30fcd92a3eb44f28ed8475204ba97 not found: ID does not exist" containerID="499b5d2794b343bfaf72afa56e0d8eaf65d30fcd92a3eb44f28ed8475204ba97" Sep 10 13:46:05.536511 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:05.536418 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"499b5d2794b343bfaf72afa56e0d8eaf65d30fcd92a3eb44f28ed8475204ba97"} err="failed to get container status \"499b5d2794b343bfaf72afa56e0d8eaf65d30fcd92a3eb44f28ed8475204ba97\": rpc error: code = NotFound desc = could not find container \"499b5d2794b343bfaf72afa56e0d8eaf65d30fcd92a3eb44f28ed8475204ba97\": container with ID starting with 499b5d2794b343bfaf72afa56e0d8eaf65d30fcd92a3eb44f28ed8475204ba97 not found: ID does not exist" Sep 10 13:46:05.536511 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:05.536437 2560 scope.go:117] "RemoveContainer" containerID="0c59d498f8b6be02d3585a14118f48105fb8892de42390d5697e5e616955155d" Sep 10 13:46:05.545254 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:05.545237 2560 scope.go:117] "RemoveContainer" containerID="0c59d498f8b6be02d3585a14118f48105fb8892de42390d5697e5e616955155d" Sep 10 13:46:05.545542 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:46:05.545521 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c59d498f8b6be02d3585a14118f48105fb8892de42390d5697e5e616955155d\": container with ID starting with 0c59d498f8b6be02d3585a14118f48105fb8892de42390d5697e5e616955155d not found: ID does not exist" containerID="0c59d498f8b6be02d3585a14118f48105fb8892de42390d5697e5e616955155d" Sep 10 13:46:05.545591 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:05.545552 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c59d498f8b6be02d3585a14118f48105fb8892de42390d5697e5e616955155d"} err="failed to get container status \"0c59d498f8b6be02d3585a14118f48105fb8892de42390d5697e5e616955155d\": rpc error: code = NotFound desc = could not find container \"0c59d498f8b6be02d3585a14118f48105fb8892de42390d5697e5e616955155d\": container with ID starting with 0c59d498f8b6be02d3585a14118f48105fb8892de42390d5697e5e616955155d not found: ID does not exist" Sep 10 13:46:05.555384 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:05.555353 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a2d945-796fb9554b-24rc4"] Sep 10 13:46:05.558182 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:05.558155 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a2d945-796fb9554b-24rc4"] Sep 10 13:46:05.570644 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:05.570613 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a2d945-c7677fc7c-b8jdd"] Sep 10 13:46:05.576640 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:05.576613 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a2d945-c7677fc7c-b8jdd"] Sep 10 13:46:06.563417 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:06.563380 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4462dd2c-1648-4139-b60c-7ee68956bfd3" path="/var/lib/kubelet/pods/4462dd2c-1648-4139-b60c-7ee68956bfd3/volumes" Sep 10 13:46:06.563799 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:06.563766 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="709f5a1a-4ce6-4b54-9b9a-b273a51a3eae" path="/var/lib/kubelet/pods/709f5a1a-4ce6-4b54-9b9a-b273a51a3eae/volumes" Sep 10 13:46:06.564081 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:06.564067 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f58519d-aee2-4789-b75b-e55117a9ab07" path="/var/lib/kubelet/pods/8f58519d-aee2-4789-b75b-e55117a9ab07/volumes" Sep 10 13:46:11.870628 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:11.870594 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:46:13.154183 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:13.154148 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:46:14.634073 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:14.634038 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:46:26.922944 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:26.922907 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:46:28.098756 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.098721 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:46:28.128303 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.128265 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-02a403-7575c48fc-ztgqf"] Sep 10 13:46:28.128615 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.128585 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-02a403-7575c48fc-ztgqf" podUID="ad82f10d-fa27-43d2-9e46-1d70e172357e" containerName="payload-processing" containerID="cri-o://db10cd17cdb217076441c4909230b204ad08c761b116067a7128af3e887e6b63" gracePeriod=30 Sep 10 13:46:28.135918 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.135888 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-02a403-5569f8fc5d-mwn4z"] Sep 10 13:46:28.136120 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.136095 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-02a403-5569f8fc5d-mwn4z" podUID="87c726c0-75d8-4960-b5e0-6be72150b117" containerName="payload-pre-processing" containerID="cri-o://9d1730d90fb21ece8407b7ad3903ca6e3b7d9634f3f777f480045ec3762f7f90" gracePeriod=30 Sep 10 13:46:28.175593 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.175560 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:46:28.253595 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.253562 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:46:28.323873 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.323841 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:46:28.388725 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.388693 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:46:28.389093 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.389077 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-02a403-7575c48fc-ztgqf" Sep 10 13:46:28.421454 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.421424 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-02a403-5569f8fc5d-mwn4z" Sep 10 13:46:28.427876 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.427853 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zsqqz\" (UniqueName: \"kubernetes.io/projected/ad82f10d-fa27-43d2-9e46-1d70e172357e-kube-api-access-zsqqz\") pod \"ad82f10d-fa27-43d2-9e46-1d70e172357e\" (UID: \"ad82f10d-fa27-43d2-9e46-1d70e172357e\") " Sep 10 13:46:28.428013 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.427948 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad82f10d-fa27-43d2-9e46-1d70e172357e-plugins-config-volume\") pod \"ad82f10d-fa27-43d2-9e46-1d70e172357e\" (UID: \"ad82f10d-fa27-43d2-9e46-1d70e172357e\") " Sep 10 13:46:28.428326 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.428302 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad82f10d-fa27-43d2-9e46-1d70e172357e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ad82f10d-fa27-43d2-9e46-1d70e172357e" (UID: "ad82f10d-fa27-43d2-9e46-1d70e172357e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:46:28.429991 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.429963 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad82f10d-fa27-43d2-9e46-1d70e172357e-kube-api-access-zsqqz" (OuterVolumeSpecName: "kube-api-access-zsqqz") pod "ad82f10d-fa27-43d2-9e46-1d70e172357e" (UID: "ad82f10d-fa27-43d2-9e46-1d70e172357e"). InnerVolumeSpecName "kube-api-access-zsqqz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:46:28.529200 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.529168 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cm5tf\" (UniqueName: \"kubernetes.io/projected/87c726c0-75d8-4960-b5e0-6be72150b117-kube-api-access-cm5tf\") pod \"87c726c0-75d8-4960-b5e0-6be72150b117\" (UID: \"87c726c0-75d8-4960-b5e0-6be72150b117\") " Sep 10 13:46:28.529374 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.529239 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87c726c0-75d8-4960-b5e0-6be72150b117-plugins-config-volume\") pod \"87c726c0-75d8-4960-b5e0-6be72150b117\" (UID: \"87c726c0-75d8-4960-b5e0-6be72150b117\") " Sep 10 13:46:28.529565 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.529550 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad82f10d-fa27-43d2-9e46-1d70e172357e-plugins-config-volume\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:46:28.529620 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.529569 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zsqqz\" (UniqueName: \"kubernetes.io/projected/ad82f10d-fa27-43d2-9e46-1d70e172357e-kube-api-access-zsqqz\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:46:28.529656 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.529621 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87c726c0-75d8-4960-b5e0-6be72150b117-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "87c726c0-75d8-4960-b5e0-6be72150b117" (UID: "87c726c0-75d8-4960-b5e0-6be72150b117"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:46:28.531287 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.531264 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87c726c0-75d8-4960-b5e0-6be72150b117-kube-api-access-cm5tf" (OuterVolumeSpecName: "kube-api-access-cm5tf") pod "87c726c0-75d8-4960-b5e0-6be72150b117" (UID: "87c726c0-75d8-4960-b5e0-6be72150b117"). InnerVolumeSpecName "kube-api-access-cm5tf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:46:28.631213 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.631113 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87c726c0-75d8-4960-b5e0-6be72150b117-plugins-config-volume\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:46:28.631213 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.631149 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cm5tf\" (UniqueName: \"kubernetes.io/projected/87c726c0-75d8-4960-b5e0-6be72150b117-kube-api-access-cm5tf\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:46:28.666223 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.666186 2560 generic.go:356] "Generic (PLEG): container finished" podID="87c726c0-75d8-4960-b5e0-6be72150b117" containerID="9d1730d90fb21ece8407b7ad3903ca6e3b7d9634f3f777f480045ec3762f7f90" exitCode=0 Sep 10 13:46:28.666437 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.666235 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-02a403-5569f8fc5d-mwn4z" Sep 10 13:46:28.666437 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.666281 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-02a403-5569f8fc5d-mwn4z" event={"ID":"87c726c0-75d8-4960-b5e0-6be72150b117","Type":"ContainerDied","Data":"9d1730d90fb21ece8407b7ad3903ca6e3b7d9634f3f777f480045ec3762f7f90"} Sep 10 13:46:28.666437 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.666344 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-02a403-5569f8fc5d-mwn4z" event={"ID":"87c726c0-75d8-4960-b5e0-6be72150b117","Type":"ContainerDied","Data":"9f1446066af5b98f8e9b13b2cdce4f2bf0378b750af74f92ae7f1cc92f2e5163"} Sep 10 13:46:28.666437 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.666375 2560 scope.go:117] "RemoveContainer" containerID="9d1730d90fb21ece8407b7ad3903ca6e3b7d9634f3f777f480045ec3762f7f90" Sep 10 13:46:28.667852 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.667831 2560 generic.go:356] "Generic (PLEG): container finished" podID="ad82f10d-fa27-43d2-9e46-1d70e172357e" containerID="db10cd17cdb217076441c4909230b204ad08c761b116067a7128af3e887e6b63" exitCode=0 Sep 10 13:46:28.667950 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.667870 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-02a403-7575c48fc-ztgqf" Sep 10 13:46:28.667950 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.667923 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-02a403-7575c48fc-ztgqf" event={"ID":"ad82f10d-fa27-43d2-9e46-1d70e172357e","Type":"ContainerDied","Data":"db10cd17cdb217076441c4909230b204ad08c761b116067a7128af3e887e6b63"} Sep 10 13:46:28.668056 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.667981 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-02a403-7575c48fc-ztgqf" event={"ID":"ad82f10d-fa27-43d2-9e46-1d70e172357e","Type":"ContainerDied","Data":"041429cdbbd045b42e88aa9ddccd1be99e04cf4fc1660f8c9afc5a55b4f33856"} Sep 10 13:46:28.676075 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.676056 2560 scope.go:117] "RemoveContainer" containerID="9d1730d90fb21ece8407b7ad3903ca6e3b7d9634f3f777f480045ec3762f7f90" Sep 10 13:46:28.676385 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:46:28.676365 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d1730d90fb21ece8407b7ad3903ca6e3b7d9634f3f777f480045ec3762f7f90\": container with ID starting with 9d1730d90fb21ece8407b7ad3903ca6e3b7d9634f3f777f480045ec3762f7f90 not found: ID does not exist" containerID="9d1730d90fb21ece8407b7ad3903ca6e3b7d9634f3f777f480045ec3762f7f90" Sep 10 13:46:28.676460 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.676410 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d1730d90fb21ece8407b7ad3903ca6e3b7d9634f3f777f480045ec3762f7f90"} err="failed to get container status \"9d1730d90fb21ece8407b7ad3903ca6e3b7d9634f3f777f480045ec3762f7f90\": rpc error: code = NotFound desc = could not find container \"9d1730d90fb21ece8407b7ad3903ca6e3b7d9634f3f777f480045ec3762f7f90\": container with ID starting with 9d1730d90fb21ece8407b7ad3903ca6e3b7d9634f3f777f480045ec3762f7f90 not found: ID does not exist" Sep 10 13:46:28.676460 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.676437 2560 scope.go:117] "RemoveContainer" containerID="db10cd17cdb217076441c4909230b204ad08c761b116067a7128af3e887e6b63" Sep 10 13:46:28.685309 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.685277 2560 scope.go:117] "RemoveContainer" containerID="db10cd17cdb217076441c4909230b204ad08c761b116067a7128af3e887e6b63" Sep 10 13:46:28.685796 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:46:28.685733 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db10cd17cdb217076441c4909230b204ad08c761b116067a7128af3e887e6b63\": container with ID starting with db10cd17cdb217076441c4909230b204ad08c761b116067a7128af3e887e6b63 not found: ID does not exist" containerID="db10cd17cdb217076441c4909230b204ad08c761b116067a7128af3e887e6b63" Sep 10 13:46:28.685911 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.685805 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db10cd17cdb217076441c4909230b204ad08c761b116067a7128af3e887e6b63"} err="failed to get container status \"db10cd17cdb217076441c4909230b204ad08c761b116067a7128af3e887e6b63\": rpc error: code = NotFound desc = could not find container \"db10cd17cdb217076441c4909230b204ad08c761b116067a7128af3e887e6b63\": container with ID starting with db10cd17cdb217076441c4909230b204ad08c761b116067a7128af3e887e6b63 not found: ID does not exist" Sep 10 13:46:28.689387 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.689366 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-02a403-5569f8fc5d-mwn4z"] Sep 10 13:46:28.692083 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.692063 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-02a403-5569f8fc5d-mwn4z"] Sep 10 13:46:28.703842 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.703808 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-02a403-7575c48fc-ztgqf"] Sep 10 13:46:28.709115 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.709074 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-02a403-7575c48fc-ztgqf"] Sep 10 13:46:28.737317 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:28.737282 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:46:29.264307 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:29.264274 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:46:30.564121 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:30.564086 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87c726c0-75d8-4960-b5e0-6be72150b117" path="/var/lib/kubelet/pods/87c726c0-75d8-4960-b5e0-6be72150b117/volumes" Sep 10 13:46:30.564512 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:30.564483 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad82f10d-fa27-43d2-9e46-1d70e172357e" path="/var/lib/kubelet/pods/ad82f10d-fa27-43d2-9e46-1d70e172357e/volumes" Sep 10 13:46:35.049457 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:35.049419 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:46:35.148469 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:35.148436 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:46:35.305038 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:35.304943 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:46:35.824993 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:35.824955 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:46:46.192753 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:46.192702 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-5d75ac-29817465-97xwj"] Sep 10 13:46:46.195847 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:46.195820 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:46:46.197314 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:46.197286 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-5d75ac-29817465-97xwj"] Sep 10 13:46:46.223038 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:46.222999 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5d75ac-57487949fd-djhg9"] Sep 10 13:46:46.223323 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:46.223287 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5d75ac-57487949fd-djhg9" podUID="338e1878-2d19-4feb-8258-f03b66237bc2" containerName="payload-pre-processing" containerID="cri-o://6e9562f49c15cd10303e59de13f93258dd5d5b7ca29f9391a560729debba5979" gracePeriod=30 Sep 10 13:46:46.267688 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:46.267656 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:46:46.367856 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:46.367818 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:46:46.470965 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:46.470870 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:46:46.503906 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:46.503877 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5d75ac-57487949fd-djhg9" Sep 10 13:46:46.563662 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:46.563629 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8160c0d1-974d-42cb-9802-49b78cf5ec43" path="/var/lib/kubelet/pods/8160c0d1-974d-42cb-9802-49b78cf5ec43/volumes" Sep 10 13:46:46.613373 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:46.613342 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/338e1878-2d19-4feb-8258-f03b66237bc2-plugins-config-volume\") pod \"338e1878-2d19-4feb-8258-f03b66237bc2\" (UID: \"338e1878-2d19-4feb-8258-f03b66237bc2\") " Sep 10 13:46:46.613579 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:46.613496 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sx6vn\" (UniqueName: \"kubernetes.io/projected/338e1878-2d19-4feb-8258-f03b66237bc2-kube-api-access-sx6vn\") pod \"338e1878-2d19-4feb-8258-f03b66237bc2\" (UID: \"338e1878-2d19-4feb-8258-f03b66237bc2\") " Sep 10 13:46:46.613755 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:46.613731 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/338e1878-2d19-4feb-8258-f03b66237bc2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "338e1878-2d19-4feb-8258-f03b66237bc2" (UID: "338e1878-2d19-4feb-8258-f03b66237bc2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:46:46.615604 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:46.615581 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/338e1878-2d19-4feb-8258-f03b66237bc2-kube-api-access-sx6vn" (OuterVolumeSpecName: "kube-api-access-sx6vn") pod "338e1878-2d19-4feb-8258-f03b66237bc2" (UID: "338e1878-2d19-4feb-8258-f03b66237bc2"). InnerVolumeSpecName "kube-api-access-sx6vn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:46:46.715123 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:46.715091 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sx6vn\" (UniqueName: \"kubernetes.io/projected/338e1878-2d19-4feb-8258-f03b66237bc2-kube-api-access-sx6vn\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:46:46.715123 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:46.715119 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/338e1878-2d19-4feb-8258-f03b66237bc2-plugins-config-volume\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:46:46.749706 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:46.749609 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:46:46.783186 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:46.783151 2560 generic.go:356] "Generic (PLEG): container finished" podID="338e1878-2d19-4feb-8258-f03b66237bc2" containerID="6e9562f49c15cd10303e59de13f93258dd5d5b7ca29f9391a560729debba5979" exitCode=0 Sep 10 13:46:46.783382 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:46.783208 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5d75ac-57487949fd-djhg9" Sep 10 13:46:46.783382 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:46.783238 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5d75ac-57487949fd-djhg9" event={"ID":"338e1878-2d19-4feb-8258-f03b66237bc2","Type":"ContainerDied","Data":"6e9562f49c15cd10303e59de13f93258dd5d5b7ca29f9391a560729debba5979"} Sep 10 13:46:46.783382 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:46.783293 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5d75ac-57487949fd-djhg9" event={"ID":"338e1878-2d19-4feb-8258-f03b66237bc2","Type":"ContainerDied","Data":"08e37360227a53b8b6ee54370398cea483a7ae7907a048499d2e0d0cb53135e4"} Sep 10 13:46:46.783382 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:46.783318 2560 scope.go:117] "RemoveContainer" containerID="6e9562f49c15cd10303e59de13f93258dd5d5b7ca29f9391a560729debba5979" Sep 10 13:46:46.793189 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:46.793169 2560 scope.go:117] "RemoveContainer" containerID="6e9562f49c15cd10303e59de13f93258dd5d5b7ca29f9391a560729debba5979" Sep 10 13:46:46.793493 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:46:46.793470 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e9562f49c15cd10303e59de13f93258dd5d5b7ca29f9391a560729debba5979\": container with ID starting with 6e9562f49c15cd10303e59de13f93258dd5d5b7ca29f9391a560729debba5979 not found: ID does not exist" containerID="6e9562f49c15cd10303e59de13f93258dd5d5b7ca29f9391a560729debba5979" Sep 10 13:46:46.793608 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:46.793502 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e9562f49c15cd10303e59de13f93258dd5d5b7ca29f9391a560729debba5979"} err="failed to get container status \"6e9562f49c15cd10303e59de13f93258dd5d5b7ca29f9391a560729debba5979\": rpc error: code = NotFound desc = could not find container \"6e9562f49c15cd10303e59de13f93258dd5d5b7ca29f9391a560729debba5979\": container with ID starting with 6e9562f49c15cd10303e59de13f93258dd5d5b7ca29f9391a560729debba5979 not found: ID does not exist" Sep 10 13:46:46.804591 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:46.804562 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5d75ac-57487949fd-djhg9"] Sep 10 13:46:46.808838 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:46.808813 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-5d75ac-57487949fd-djhg9"] Sep 10 13:46:48.564774 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:48.564737 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="338e1878-2d19-4feb-8258-f03b66237bc2" path="/var/lib/kubelet/pods/338e1878-2d19-4feb-8258-f03b66237bc2/volumes" Sep 10 13:46:53.266791 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:53.266761 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:46:53.339018 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:53.338986 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:46:53.531317 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:53.531240 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:46:54.087976 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:54.087943 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:46:57.553916 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:57.553882 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:46:59.399173 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:59.399127 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:46:59.488477 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:59.488429 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:46:59.647204 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:46:59.647162 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:47:02.598015 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:02.597981 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:47:02.700142 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:02.700100 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:47:03.897807 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:03.897775 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:47:04.000984 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:04.000953 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:47:04.424270 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:04.424233 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:47:04.549098 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:04.549026 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:47:04.650384 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:04.650332 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:47:04.667754 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:04.667711 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-41abaf-56d9cc7b59-gmlzl"] Sep 10 13:47:04.668247 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:04.668226 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad82f10d-fa27-43d2-9e46-1d70e172357e" containerName="payload-processing" Sep 10 13:47:04.668247 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:04.668246 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad82f10d-fa27-43d2-9e46-1d70e172357e" containerName="payload-processing" Sep 10 13:47:04.668456 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:04.668259 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="338e1878-2d19-4feb-8258-f03b66237bc2" containerName="payload-pre-processing" Sep 10 13:47:04.668456 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:04.668268 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="338e1878-2d19-4feb-8258-f03b66237bc2" containerName="payload-pre-processing" Sep 10 13:47:04.668456 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:04.668284 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87c726c0-75d8-4960-b5e0-6be72150b117" containerName="payload-pre-processing" Sep 10 13:47:04.668456 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:04.668292 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="87c726c0-75d8-4960-b5e0-6be72150b117" containerName="payload-pre-processing" Sep 10 13:47:04.668456 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:04.668322 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f58519d-aee2-4789-b75b-e55117a9ab07" containerName="payload-pre-processing" Sep 10 13:47:04.668456 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:04.668330 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f58519d-aee2-4789-b75b-e55117a9ab07" containerName="payload-pre-processing" Sep 10 13:47:04.668456 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:04.668350 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4462dd2c-1648-4139-b60c-7ee68956bfd3" containerName="payload-processing" Sep 10 13:47:04.668456 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:04.668358 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4462dd2c-1648-4139-b60c-7ee68956bfd3" containerName="payload-processing" Sep 10 13:47:04.668817 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:04.668468 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4462dd2c-1648-4139-b60c-7ee68956bfd3" containerName="payload-processing" Sep 10 13:47:04.668817 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:04.668479 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="338e1878-2d19-4feb-8258-f03b66237bc2" containerName="payload-pre-processing" Sep 10 13:47:04.668817 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:04.668487 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="87c726c0-75d8-4960-b5e0-6be72150b117" containerName="payload-pre-processing" Sep 10 13:47:04.668817 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:04.668497 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad82f10d-fa27-43d2-9e46-1d70e172357e" containerName="payload-processing" Sep 10 13:47:04.668817 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:04.668506 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f58519d-aee2-4789-b75b-e55117a9ab07" containerName="payload-pre-processing" Sep 10 13:47:04.673154 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:04.673132 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-41abaf-56d9cc7b59-gmlzl" Sep 10 13:47:04.676315 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:04.676247 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-41abaf\"" Sep 10 13:47:04.676466 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:04.676310 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-41abaf-dockercfg-r5hlz\"" Sep 10 13:47:04.686325 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:04.686299 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-41abaf-56d9cc7b59-gmlzl"] Sep 10 13:47:04.790505 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:04.790454 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/796ef08b-1b48-4811-8a48-1aa630675f0e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-41abaf-56d9cc7b59-gmlzl\" (UID: \"796ef08b-1b48-4811-8a48-1aa630675f0e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-41abaf-56d9cc7b59-gmlzl" Sep 10 13:47:04.790700 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:04.790626 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxnjs\" (UniqueName: \"kubernetes.io/projected/796ef08b-1b48-4811-8a48-1aa630675f0e-kube-api-access-kxnjs\") pod \"payload-processing-e2e-ipp-a-41abaf-56d9cc7b59-gmlzl\" (UID: \"796ef08b-1b48-4811-8a48-1aa630675f0e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-41abaf-56d9cc7b59-gmlzl" Sep 10 13:47:04.885729 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:04.885693 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:47:04.886291 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:04.886273 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-ba174e-29817465-fdl7c"] Sep 10 13:47:04.888796 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:04.888775 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-ba174e-29817465-fdl7c"] Sep 10 13:47:04.891125 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:04.891106 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/796ef08b-1b48-4811-8a48-1aa630675f0e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-41abaf-56d9cc7b59-gmlzl\" (UID: \"796ef08b-1b48-4811-8a48-1aa630675f0e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-41abaf-56d9cc7b59-gmlzl" Sep 10 13:47:04.891228 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:04.891218 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxnjs\" (UniqueName: \"kubernetes.io/projected/796ef08b-1b48-4811-8a48-1aa630675f0e-kube-api-access-kxnjs\") pod \"payload-processing-e2e-ipp-a-41abaf-56d9cc7b59-gmlzl\" (UID: \"796ef08b-1b48-4811-8a48-1aa630675f0e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-41abaf-56d9cc7b59-gmlzl" Sep 10 13:47:04.896425 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:04.892332 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/796ef08b-1b48-4811-8a48-1aa630675f0e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-41abaf-56d9cc7b59-gmlzl\" (UID: \"796ef08b-1b48-4811-8a48-1aa630675f0e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-41abaf-56d9cc7b59-gmlzl" Sep 10 13:47:04.902982 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:04.902950 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxnjs\" (UniqueName: \"kubernetes.io/projected/796ef08b-1b48-4811-8a48-1aa630675f0e-kube-api-access-kxnjs\") pod \"payload-processing-e2e-ipp-a-41abaf-56d9cc7b59-gmlzl\" (UID: \"796ef08b-1b48-4811-8a48-1aa630675f0e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-41abaf-56d9cc7b59-gmlzl" Sep 10 13:47:04.919489 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:04.919444 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ba174e-7c68c6657d-7l78v"] Sep 10 13:47:04.919786 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:04.919727 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ba174e-7c68c6657d-7l78v" podUID="88a911a9-b2dc-400f-b27c-4cca66213ef9" containerName="payload-pre-processing" containerID="cri-o://8ca8122feda20997afaaa9d333064d8d2050b244215ce6956e4951db1a5f0f05" gracePeriod=30 Sep 10 13:47:04.987078 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:04.987042 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-41abaf-56d9cc7b59-gmlzl" Sep 10 13:47:04.987545 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:04.987505 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:47:05.065060 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:05.064948 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:47:05.143582 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:05.143550 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:47:05.239147 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:05.239114 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:47:05.242491 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:05.242423 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ba174e-7c68c6657d-7l78v" Sep 10 13:47:05.279189 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:05.279137 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-41abaf-56d9cc7b59-gmlzl"] Sep 10 13:47:05.279820 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:47:05.279797 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod796ef08b_1b48_4811_8a48_1aa630675f0e.slice/crio-2c9a614205100cee55b00a7fd4aba3294b158d72413571a4e70be3f7f5832feb WatchSource:0}: Error finding container 2c9a614205100cee55b00a7fd4aba3294b158d72413571a4e70be3f7f5832feb: Status 404 returned error can't find the container with id 2c9a614205100cee55b00a7fd4aba3294b158d72413571a4e70be3f7f5832feb Sep 10 13:47:05.305468 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:05.305440 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88a911a9-b2dc-400f-b27c-4cca66213ef9-plugins-config-volume\") pod \"88a911a9-b2dc-400f-b27c-4cca66213ef9\" (UID: \"88a911a9-b2dc-400f-b27c-4cca66213ef9\") " Sep 10 13:47:05.305615 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:05.305524 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5wc5c\" (UniqueName: \"kubernetes.io/projected/88a911a9-b2dc-400f-b27c-4cca66213ef9-kube-api-access-5wc5c\") pod \"88a911a9-b2dc-400f-b27c-4cca66213ef9\" (UID: \"88a911a9-b2dc-400f-b27c-4cca66213ef9\") " Sep 10 13:47:05.305789 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:05.305762 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88a911a9-b2dc-400f-b27c-4cca66213ef9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "88a911a9-b2dc-400f-b27c-4cca66213ef9" (UID: "88a911a9-b2dc-400f-b27c-4cca66213ef9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:47:05.307551 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:05.307526 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88a911a9-b2dc-400f-b27c-4cca66213ef9-kube-api-access-5wc5c" (OuterVolumeSpecName: "kube-api-access-5wc5c") pod "88a911a9-b2dc-400f-b27c-4cca66213ef9" (UID: "88a911a9-b2dc-400f-b27c-4cca66213ef9"). InnerVolumeSpecName "kube-api-access-5wc5c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:47:05.406632 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:05.406592 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88a911a9-b2dc-400f-b27c-4cca66213ef9-plugins-config-volume\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:47:05.406632 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:05.406626 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wc5c\" (UniqueName: \"kubernetes.io/projected/88a911a9-b2dc-400f-b27c-4cca66213ef9-kube-api-access-5wc5c\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:47:05.525553 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:05.525432 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:47:05.912107 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:05.911555 2560 generic.go:356] "Generic (PLEG): container finished" podID="88a911a9-b2dc-400f-b27c-4cca66213ef9" containerID="8ca8122feda20997afaaa9d333064d8d2050b244215ce6956e4951db1a5f0f05" exitCode=0 Sep 10 13:47:05.912107 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:05.911616 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ba174e-7c68c6657d-7l78v" Sep 10 13:47:05.912107 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:05.911663 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ba174e-7c68c6657d-7l78v" event={"ID":"88a911a9-b2dc-400f-b27c-4cca66213ef9","Type":"ContainerDied","Data":"8ca8122feda20997afaaa9d333064d8d2050b244215ce6956e4951db1a5f0f05"} Sep 10 13:47:05.912107 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:05.911736 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ba174e-7c68c6657d-7l78v" event={"ID":"88a911a9-b2dc-400f-b27c-4cca66213ef9","Type":"ContainerDied","Data":"04cea5d0abf73c7aad315098e4731348085ac53259ec4558cc75e7047b62f076"} Sep 10 13:47:05.912107 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:05.911768 2560 scope.go:117] "RemoveContainer" containerID="8ca8122feda20997afaaa9d333064d8d2050b244215ce6956e4951db1a5f0f05" Sep 10 13:47:05.914684 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:05.914655 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-41abaf-56d9cc7b59-gmlzl" event={"ID":"796ef08b-1b48-4811-8a48-1aa630675f0e","Type":"ContainerStarted","Data":"90470bf3c972be33b10f4dcb4d0d403bf38a4f40ec301ad8d38bf00f13294ce9"} Sep 10 13:47:05.914816 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:05.914713 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-41abaf-56d9cc7b59-gmlzl" event={"ID":"796ef08b-1b48-4811-8a48-1aa630675f0e","Type":"ContainerStarted","Data":"2c9a614205100cee55b00a7fd4aba3294b158d72413571a4e70be3f7f5832feb"} Sep 10 13:47:05.914884 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:05.914821 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-41abaf-56d9cc7b59-gmlzl" Sep 10 13:47:05.923802 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:05.923634 2560 scope.go:117] "RemoveContainer" containerID="8ca8122feda20997afaaa9d333064d8d2050b244215ce6956e4951db1a5f0f05" Sep 10 13:47:05.923974 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:47:05.923953 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ca8122feda20997afaaa9d333064d8d2050b244215ce6956e4951db1a5f0f05\": container with ID starting with 8ca8122feda20997afaaa9d333064d8d2050b244215ce6956e4951db1a5f0f05 not found: ID does not exist" containerID="8ca8122feda20997afaaa9d333064d8d2050b244215ce6956e4951db1a5f0f05" Sep 10 13:47:05.924038 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:05.923986 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ca8122feda20997afaaa9d333064d8d2050b244215ce6956e4951db1a5f0f05"} err="failed to get container status \"8ca8122feda20997afaaa9d333064d8d2050b244215ce6956e4951db1a5f0f05\": rpc error: code = NotFound desc = could not find container \"8ca8122feda20997afaaa9d333064d8d2050b244215ce6956e4951db1a5f0f05\": container with ID starting with 8ca8122feda20997afaaa9d333064d8d2050b244215ce6956e4951db1a5f0f05 not found: ID does not exist" Sep 10 13:47:05.937040 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:05.936992 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-41abaf-56d9cc7b59-gmlzl" podStartSLOduration=1.936976121 podStartE2EDuration="1.936976121s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:47:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:47:05.935174735 +0000 UTC m=+1735.829892150" watchObservedRunningTime="2026-09-10 13:47:05.936976121 +0000 UTC m=+1735.831693539" Sep 10 13:47:05.951563 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:05.951529 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ba174e-7c68c6657d-7l78v"] Sep 10 13:47:05.961996 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:05.961963 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-ba174e-7c68c6657d-7l78v"] Sep 10 13:47:06.564903 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:06.564865 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88a911a9-b2dc-400f-b27c-4cca66213ef9" path="/var/lib/kubelet/pods/88a911a9-b2dc-400f-b27c-4cca66213ef9/volumes" Sep 10 13:47:06.565273 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:06.565253 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4e27dd4-c5c8-4bea-a236-07fb72a40da2" path="/var/lib/kubelet/pods/e4e27dd4-c5c8-4bea-a236-07fb72a40da2/volumes" Sep 10 13:47:12.072134 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:12.072093 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:47:12.192033 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:12.192000 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:47:12.305629 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:12.305587 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj"] Sep 10 13:47:12.306168 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:12.305871 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" podUID="6a1f48da-7c79-4e1f-a9e5-6371c4445a38" containerName="istio-proxy" containerID="cri-o://abe819dba1436d392d6a02e30550282b7172d4cbb9c62b681e068c2793779e9d" gracePeriod=30 Sep 10 13:47:12.334789 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:12.334706 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:47:16.924754 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:16.924718 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-41abaf-56d9cc7b59-gmlzl" Sep 10 13:47:17.567189 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:17.567118 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:47:17.734580 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:17.734538 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-istio-token\") pod \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\" (UID: \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\") " Sep 10 13:47:17.734785 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:17.734597 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xzhsr\" (UniqueName: \"kubernetes.io/projected/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-kube-api-access-xzhsr\") pod \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\" (UID: \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\") " Sep 10 13:47:17.734785 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:17.734620 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-istio-envoy\") pod \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\" (UID: \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\") " Sep 10 13:47:17.734785 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:17.734660 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-workload-certs\") pod \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\" (UID: \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\") " Sep 10 13:47:17.734785 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:17.734711 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-workload-socket\") pod \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\" (UID: \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\") " Sep 10 13:47:17.734785 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:17.734759 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-istio-podinfo\") pod \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\" (UID: \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\") " Sep 10 13:47:17.735055 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:17.734828 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-istiod-ca-cert\") pod \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\" (UID: \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\") " Sep 10 13:47:17.735055 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:17.734877 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-istio-data\") pod \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\" (UID: \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\") " Sep 10 13:47:17.735055 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:17.734930 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-credential-socket\") pod \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\" (UID: \"6a1f48da-7c79-4e1f-a9e5-6371c4445a38\") " Sep 10 13:47:17.735218 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:17.735066 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6a1f48da-7c79-4e1f-a9e5-6371c4445a38" (UID: "6a1f48da-7c79-4e1f-a9e5-6371c4445a38"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:47:17.735218 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:17.735053 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6a1f48da-7c79-4e1f-a9e5-6371c4445a38" (UID: "6a1f48da-7c79-4e1f-a9e5-6371c4445a38"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:47:17.735330 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:17.735260 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-istio-data" (OuterVolumeSpecName: "istio-data") pod "6a1f48da-7c79-4e1f-a9e5-6371c4445a38" (UID: "6a1f48da-7c79-4e1f-a9e5-6371c4445a38"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:47:17.735330 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:17.735290 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6a1f48da-7c79-4e1f-a9e5-6371c4445a38" (UID: "6a1f48da-7c79-4e1f-a9e5-6371c4445a38"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:47:17.735330 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:17.735297 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6a1f48da-7c79-4e1f-a9e5-6371c4445a38" (UID: "6a1f48da-7c79-4e1f-a9e5-6371c4445a38"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:47:17.735533 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:17.735376 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-workload-certs\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:47:17.735533 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:17.735428 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-workload-socket\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:47:17.736887 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:17.736863 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6a1f48da-7c79-4e1f-a9e5-6371c4445a38" (UID: "6a1f48da-7c79-4e1f-a9e5-6371c4445a38"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:47:17.737009 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:17.736994 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-istio-token" (OuterVolumeSpecName: "istio-token") pod "6a1f48da-7c79-4e1f-a9e5-6371c4445a38" (UID: "6a1f48da-7c79-4e1f-a9e5-6371c4445a38"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:47:17.737506 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:17.737486 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6a1f48da-7c79-4e1f-a9e5-6371c4445a38" (UID: "6a1f48da-7c79-4e1f-a9e5-6371c4445a38"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 13:47:17.737578 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:17.737485 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-kube-api-access-xzhsr" (OuterVolumeSpecName: "kube-api-access-xzhsr") pod "6a1f48da-7c79-4e1f-a9e5-6371c4445a38" (UID: "6a1f48da-7c79-4e1f-a9e5-6371c4445a38"). InnerVolumeSpecName "kube-api-access-xzhsr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:47:17.836786 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:17.836694 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-credential-socket\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:47:17.836786 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:17.836727 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-istio-token\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:47:17.836786 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:17.836743 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzhsr\" (UniqueName: \"kubernetes.io/projected/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-kube-api-access-xzhsr\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:47:17.836786 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:17.836755 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-istio-envoy\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:47:17.836786 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:17.836767 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-istio-podinfo\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:47:17.836786 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:17.836779 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-istiod-ca-cert\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:47:17.837101 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:17.836791 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a1f48da-7c79-4e1f-a9e5-6371c4445a38-istio-data\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:47:17.988225 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:17.988194 2560 generic.go:356] "Generic (PLEG): container finished" podID="6a1f48da-7c79-4e1f-a9e5-6371c4445a38" containerID="abe819dba1436d392d6a02e30550282b7172d4cbb9c62b681e068c2793779e9d" exitCode=0 Sep 10 13:47:17.988649 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:17.988235 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" Sep 10 13:47:17.988649 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:17.988276 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" event={"ID":"6a1f48da-7c79-4e1f-a9e5-6371c4445a38","Type":"ContainerDied","Data":"abe819dba1436d392d6a02e30550282b7172d4cbb9c62b681e068c2793779e9d"} Sep 10 13:47:17.988649 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:17.988328 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj" event={"ID":"6a1f48da-7c79-4e1f-a9e5-6371c4445a38","Type":"ContainerDied","Data":"08cc1f76563116dc8fda1f784728feeb4c864e7fff5d06af5af8e71f29f31f36"} Sep 10 13:47:17.988649 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:17.988355 2560 scope.go:117] "RemoveContainer" containerID="abe819dba1436d392d6a02e30550282b7172d4cbb9c62b681e068c2793779e9d" Sep 10 13:47:17.998422 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:17.998387 2560 scope.go:117] "RemoveContainer" containerID="0605f2a309df51b17a2b06717bc2c267deb2bc96b2161554bae5abf725d5c07f" Sep 10 13:47:18.007469 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:18.007449 2560 scope.go:117] "RemoveContainer" containerID="abe819dba1436d392d6a02e30550282b7172d4cbb9c62b681e068c2793779e9d" Sep 10 13:47:18.007754 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:47:18.007732 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abe819dba1436d392d6a02e30550282b7172d4cbb9c62b681e068c2793779e9d\": container with ID starting with abe819dba1436d392d6a02e30550282b7172d4cbb9c62b681e068c2793779e9d not found: ID does not exist" containerID="abe819dba1436d392d6a02e30550282b7172d4cbb9c62b681e068c2793779e9d" Sep 10 13:47:18.007809 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:18.007764 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abe819dba1436d392d6a02e30550282b7172d4cbb9c62b681e068c2793779e9d"} err="failed to get container status \"abe819dba1436d392d6a02e30550282b7172d4cbb9c62b681e068c2793779e9d\": rpc error: code = NotFound desc = could not find container \"abe819dba1436d392d6a02e30550282b7172d4cbb9c62b681e068c2793779e9d\": container with ID starting with abe819dba1436d392d6a02e30550282b7172d4cbb9c62b681e068c2793779e9d not found: ID does not exist" Sep 10 13:47:18.007809 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:18.007781 2560 scope.go:117] "RemoveContainer" containerID="0605f2a309df51b17a2b06717bc2c267deb2bc96b2161554bae5abf725d5c07f" Sep 10 13:47:18.007982 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:47:18.007964 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0605f2a309df51b17a2b06717bc2c267deb2bc96b2161554bae5abf725d5c07f\": container with ID starting with 0605f2a309df51b17a2b06717bc2c267deb2bc96b2161554bae5abf725d5c07f not found: ID does not exist" containerID="0605f2a309df51b17a2b06717bc2c267deb2bc96b2161554bae5abf725d5c07f" Sep 10 13:47:18.008025 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:18.007986 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0605f2a309df51b17a2b06717bc2c267deb2bc96b2161554bae5abf725d5c07f"} err="failed to get container status \"0605f2a309df51b17a2b06717bc2c267deb2bc96b2161554bae5abf725d5c07f\": rpc error: code = NotFound desc = could not find container \"0605f2a309df51b17a2b06717bc2c267deb2bc96b2161554bae5abf725d5c07f\": container with ID starting with 0605f2a309df51b17a2b06717bc2c267deb2bc96b2161554bae5abf725d5c07f not found: ID does not exist" Sep 10 13:47:18.030140 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:18.030109 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj"] Sep 10 13:47:18.034478 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:18.034451 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-ba174e-openshift-default-6545f65688-wnrxj"] Sep 10 13:47:18.563027 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:18.562982 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a1f48da-7c79-4e1f-a9e5-6371c4445a38" path="/var/lib/kubelet/pods/6a1f48da-7c79-4e1f-a9e5-6371c4445a38/volumes" Sep 10 13:47:27.578142 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:27.578107 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:47:28.256039 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:28.256005 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:47:28.600198 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:28.600110 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:47:28.776841 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:28.776806 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:47:31.815200 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:31.815169 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:47:31.913181 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:31.913147 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:47:33.031549 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:33.031511 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:47:33.129957 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:33.129924 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:47:33.575657 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:33.575607 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:47:33.706291 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:33.706239 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:47:33.783287 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:33.783254 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-645758-d9d5fcd4b-pxsz5"] Sep 10 13:47:33.783760 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:33.783743 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88a911a9-b2dc-400f-b27c-4cca66213ef9" containerName="payload-pre-processing" Sep 10 13:47:33.783760 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:33.783761 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="88a911a9-b2dc-400f-b27c-4cca66213ef9" containerName="payload-pre-processing" Sep 10 13:47:33.783881 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:33.783776 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a1f48da-7c79-4e1f-a9e5-6371c4445a38" containerName="istio-proxy" Sep 10 13:47:33.783881 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:33.783783 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a1f48da-7c79-4e1f-a9e5-6371c4445a38" containerName="istio-proxy" Sep 10 13:47:33.783881 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:33.783803 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a1f48da-7c79-4e1f-a9e5-6371c4445a38" containerName="istio-proxy" Sep 10 13:47:33.783881 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:33.783809 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a1f48da-7c79-4e1f-a9e5-6371c4445a38" containerName="istio-proxy" Sep 10 13:47:33.783881 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:33.783877 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a1f48da-7c79-4e1f-a9e5-6371c4445a38" containerName="istio-proxy" Sep 10 13:47:33.784103 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:33.783892 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="88a911a9-b2dc-400f-b27c-4cca66213ef9" containerName="payload-pre-processing" Sep 10 13:47:33.788091 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:33.788074 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-645758-d9d5fcd4b-pxsz5" Sep 10 13:47:33.792209 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:33.792187 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-645758\"" Sep 10 13:47:33.792322 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:33.792292 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-645758-dockercfg-q79ph\"" Sep 10 13:47:33.802578 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:33.802551 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-645758-d9d5fcd4b-pxsz5"] Sep 10 13:47:33.885996 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:33.885907 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17221ec2-5ef2-4ef3-9889-32eb821aff1d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-645758-d9d5fcd4b-pxsz5\" (UID: \"17221ec2-5ef2-4ef3-9889-32eb821aff1d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-645758-d9d5fcd4b-pxsz5" Sep 10 13:47:33.886182 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:33.886015 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7tbh\" (UniqueName: \"kubernetes.io/projected/17221ec2-5ef2-4ef3-9889-32eb821aff1d-kube-api-access-x7tbh\") pod \"payload-pre-processing-e2e-ipp-b-645758-d9d5fcd4b-pxsz5\" (UID: \"17221ec2-5ef2-4ef3-9889-32eb821aff1d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-645758-d9d5fcd4b-pxsz5" Sep 10 13:47:33.987510 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:33.987464 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17221ec2-5ef2-4ef3-9889-32eb821aff1d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-645758-d9d5fcd4b-pxsz5\" (UID: \"17221ec2-5ef2-4ef3-9889-32eb821aff1d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-645758-d9d5fcd4b-pxsz5" Sep 10 13:47:33.987715 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:33.987545 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7tbh\" (UniqueName: \"kubernetes.io/projected/17221ec2-5ef2-4ef3-9889-32eb821aff1d-kube-api-access-x7tbh\") pod \"payload-pre-processing-e2e-ipp-b-645758-d9d5fcd4b-pxsz5\" (UID: \"17221ec2-5ef2-4ef3-9889-32eb821aff1d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-645758-d9d5fcd4b-pxsz5" Sep 10 13:47:33.988182 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:33.988160 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17221ec2-5ef2-4ef3-9889-32eb821aff1d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-645758-d9d5fcd4b-pxsz5\" (UID: \"17221ec2-5ef2-4ef3-9889-32eb821aff1d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-645758-d9d5fcd4b-pxsz5" Sep 10 13:47:33.996961 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:33.996926 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7tbh\" (UniqueName: \"kubernetes.io/projected/17221ec2-5ef2-4ef3-9889-32eb821aff1d-kube-api-access-x7tbh\") pod \"payload-pre-processing-e2e-ipp-b-645758-d9d5fcd4b-pxsz5\" (UID: \"17221ec2-5ef2-4ef3-9889-32eb821aff1d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-645758-d9d5fcd4b-pxsz5" Sep 10 13:47:34.098532 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:34.098496 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-645758-d9d5fcd4b-pxsz5" Sep 10 13:47:34.380581 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:34.380546 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-645758-d9d5fcd4b-pxsz5"] Sep 10 13:47:34.384447 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:47:34.384417 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17221ec2_5ef2_4ef3_9889_32eb821aff1d.slice/crio-12739fab42d17594bf9be869a1a7e4d55b6d320f1c09676e72fb79018d57575d WatchSource:0}: Error finding container 12739fab42d17594bf9be869a1a7e4d55b6d320f1c09676e72fb79018d57575d: Status 404 returned error can't find the container with id 12739fab42d17594bf9be869a1a7e4d55b6d320f1c09676e72fb79018d57575d Sep 10 13:47:35.091934 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:35.091899 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-645758-d9d5fcd4b-pxsz5" event={"ID":"17221ec2-5ef2-4ef3-9889-32eb821aff1d","Type":"ContainerStarted","Data":"92148123fd228359cde5da8fc8e24e5a1abeb04f3ae7726e55cfc0b37d5a2eee"} Sep 10 13:47:35.092146 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:35.091953 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-645758-d9d5fcd4b-pxsz5" event={"ID":"17221ec2-5ef2-4ef3-9889-32eb821aff1d","Type":"ContainerStarted","Data":"12739fab42d17594bf9be869a1a7e4d55b6d320f1c09676e72fb79018d57575d"} Sep 10 13:47:35.092146 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:35.091979 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-645758-d9d5fcd4b-pxsz5" Sep 10 13:47:35.115779 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:35.115725 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-645758-d9d5fcd4b-pxsz5" podStartSLOduration=2.115708769 podStartE2EDuration="2.115708769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:47:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:47:35.114183764 +0000 UTC m=+1765.008901181" watchObservedRunningTime="2026-09-10 13:47:35.115708769 +0000 UTC m=+1765.010426207" Sep 10 13:47:36.743491 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:36.743457 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:47:46.100584 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:46.100551 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-645758-d9d5fcd4b-pxsz5" Sep 10 13:47:56.854154 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:56.854112 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:47:58.235889 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:47:58.235857 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:48:03.818198 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:03.818097 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:48:10.640222 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:10.640193 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gtqlh_52fceedd-00f2-4943-ae47-dec51216d8e4/console-operator/2.log" Sep 10 13:48:10.649883 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:10.649857 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gtqlh_52fceedd-00f2-4943-ae47-dec51216d8e4/console-operator/2.log" Sep 10 13:48:27.497350 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:27.497311 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-41abaf-56d9cc7b59-gmlzl_796ef08b-1b48-4811-8a48-1aa630675f0e/payload-processing/0.log" Sep 10 13:48:28.083164 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.083118 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:48:28.372188 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.372096 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj"] Sep 10 13:48:28.372699 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.372681 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a1f48da-7c79-4e1f-a9e5-6371c4445a38" containerName="istio-proxy" Sep 10 13:48:28.376148 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.376123 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:48:28.379416 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.379385 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-08e484-openshift-default-dockercfg-8jmgz\"" Sep 10 13:48:28.393692 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.393655 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37174f02-8469-445e-be70-3bed424acbe2-istio-data\") pod \"e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj\" (UID: \"37174f02-8469-445e-be70-3bed424acbe2\") " pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:48:28.393851 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.393713 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37174f02-8469-445e-be70-3bed424acbe2-istio-podinfo\") pod \"e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj\" (UID: \"37174f02-8469-445e-be70-3bed424acbe2\") " pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:48:28.393851 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.393794 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37174f02-8469-445e-be70-3bed424acbe2-istio-token\") pod \"e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj\" (UID: \"37174f02-8469-445e-be70-3bed424acbe2\") " pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:48:28.393851 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.393824 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37174f02-8469-445e-be70-3bed424acbe2-workload-certs\") pod \"e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj\" (UID: \"37174f02-8469-445e-be70-3bed424acbe2\") " pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:48:28.393851 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.393848 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37174f02-8469-445e-be70-3bed424acbe2-istiod-ca-cert\") pod \"e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj\" (UID: \"37174f02-8469-445e-be70-3bed424acbe2\") " pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:48:28.394043 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.393959 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37174f02-8469-445e-be70-3bed424acbe2-credential-socket\") pod \"e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj\" (UID: \"37174f02-8469-445e-be70-3bed424acbe2\") " pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:48:28.394043 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.393991 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj"] Sep 10 13:48:28.394043 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.394032 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37174f02-8469-445e-be70-3bed424acbe2-workload-socket\") pod \"e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj\" (UID: \"37174f02-8469-445e-be70-3bed424acbe2\") " pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:48:28.394191 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.394064 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37174f02-8469-445e-be70-3bed424acbe2-istio-envoy\") pod \"e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj\" (UID: \"37174f02-8469-445e-be70-3bed424acbe2\") " pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:48:28.394191 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.394097 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kr962\" (UniqueName: \"kubernetes.io/projected/37174f02-8469-445e-be70-3bed424acbe2-kube-api-access-kr962\") pod \"e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj\" (UID: \"37174f02-8469-445e-be70-3bed424acbe2\") " pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:48:28.414167 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.414130 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:48:28.495512 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.495470 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37174f02-8469-445e-be70-3bed424acbe2-credential-socket\") pod \"e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj\" (UID: \"37174f02-8469-445e-be70-3bed424acbe2\") " pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:48:28.495699 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.495554 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37174f02-8469-445e-be70-3bed424acbe2-workload-socket\") pod \"e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj\" (UID: \"37174f02-8469-445e-be70-3bed424acbe2\") " pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:48:28.495699 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.495588 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37174f02-8469-445e-be70-3bed424acbe2-istio-envoy\") pod \"e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj\" (UID: \"37174f02-8469-445e-be70-3bed424acbe2\") " pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:48:28.495699 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.495622 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr962\" (UniqueName: \"kubernetes.io/projected/37174f02-8469-445e-be70-3bed424acbe2-kube-api-access-kr962\") pod \"e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj\" (UID: \"37174f02-8469-445e-be70-3bed424acbe2\") " pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:48:28.495699 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.495684 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37174f02-8469-445e-be70-3bed424acbe2-istio-data\") pod \"e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj\" (UID: \"37174f02-8469-445e-be70-3bed424acbe2\") " pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:48:28.495914 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.495718 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37174f02-8469-445e-be70-3bed424acbe2-istio-podinfo\") pod \"e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj\" (UID: \"37174f02-8469-445e-be70-3bed424acbe2\") " pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:48:28.495914 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.495772 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37174f02-8469-445e-be70-3bed424acbe2-istio-token\") pod \"e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj\" (UID: \"37174f02-8469-445e-be70-3bed424acbe2\") " pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:48:28.495914 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.495803 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37174f02-8469-445e-be70-3bed424acbe2-workload-certs\") pod \"e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj\" (UID: \"37174f02-8469-445e-be70-3bed424acbe2\") " pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:48:28.495914 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.495827 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37174f02-8469-445e-be70-3bed424acbe2-istiod-ca-cert\") pod \"e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj\" (UID: \"37174f02-8469-445e-be70-3bed424acbe2\") " pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:48:28.496123 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.495934 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37174f02-8469-445e-be70-3bed424acbe2-credential-socket\") pod \"e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj\" (UID: \"37174f02-8469-445e-be70-3bed424acbe2\") " pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:48:28.496123 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.495945 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37174f02-8469-445e-be70-3bed424acbe2-workload-socket\") pod \"e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj\" (UID: \"37174f02-8469-445e-be70-3bed424acbe2\") " pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:48:28.496237 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.496206 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37174f02-8469-445e-be70-3bed424acbe2-istio-data\") pod \"e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj\" (UID: \"37174f02-8469-445e-be70-3bed424acbe2\") " pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:48:28.496329 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.496301 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37174f02-8469-445e-be70-3bed424acbe2-workload-certs\") pod \"e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj\" (UID: \"37174f02-8469-445e-be70-3bed424acbe2\") " pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:48:28.496610 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.496585 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37174f02-8469-445e-be70-3bed424acbe2-istiod-ca-cert\") pod \"e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj\" (UID: \"37174f02-8469-445e-be70-3bed424acbe2\") " pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:48:28.499058 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.499033 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37174f02-8469-445e-be70-3bed424acbe2-istio-podinfo\") pod \"e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj\" (UID: \"37174f02-8469-445e-be70-3bed424acbe2\") " pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:48:28.499368 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.499137 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37174f02-8469-445e-be70-3bed424acbe2-istio-envoy\") pod \"e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj\" (UID: \"37174f02-8469-445e-be70-3bed424acbe2\") " pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:48:28.505559 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.505531 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37174f02-8469-445e-be70-3bed424acbe2-istio-token\") pod \"e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj\" (UID: \"37174f02-8469-445e-be70-3bed424acbe2\") " pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:48:28.506002 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.505978 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kr962\" (UniqueName: \"kubernetes.io/projected/37174f02-8469-445e-be70-3bed424acbe2-kube-api-access-kr962\") pod \"e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj\" (UID: \"37174f02-8469-445e-be70-3bed424acbe2\") " pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:48:28.688277 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.688239 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:48:28.781376 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.781340 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:48:28.973673 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.973639 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj"] Sep 10 13:48:28.974260 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:48:28.974235 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37174f02_8469_445e_be70_3bed424acbe2.slice/crio-5a5e3831c1f304027dd2d5979ea45a7770a7b13d239d3b426c8475794f55add1 WatchSource:0}: Error finding container 5a5e3831c1f304027dd2d5979ea45a7770a7b13d239d3b426c8475794f55add1: Status 404 returned error can't find the container with id 5a5e3831c1f304027dd2d5979ea45a7770a7b13d239d3b426c8475794f55add1 Sep 10 13:48:28.976712 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.976677 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:48:28.976803 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.976762 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:48:28.976842 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:28.976813 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:48:29.423352 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:29.423314 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" event={"ID":"37174f02-8469-445e-be70-3bed424acbe2","Type":"ContainerStarted","Data":"ee61dc866367f78bab0058e7913d047016dab9af9afc873cfa2b0b5e70131945"} Sep 10 13:48:29.423531 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:29.423370 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" event={"ID":"37174f02-8469-445e-be70-3bed424acbe2","Type":"ContainerStarted","Data":"5a5e3831c1f304027dd2d5979ea45a7770a7b13d239d3b426c8475794f55add1"} Sep 10 13:48:29.451049 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:29.450979 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" podStartSLOduration=1.450955398 podStartE2EDuration="1.450955398s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:48:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:48:29.448078668 +0000 UTC m=+1819.342796103" watchObservedRunningTime="2026-09-10 13:48:29.450955398 +0000 UTC m=+1819.345672816" Sep 10 13:48:29.689467 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:29.689350 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:48:30.694278 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:30.694245 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:48:31.436014 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:31.435979 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:48:31.437055 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:31.437034 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:48:31.451256 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:31.451218 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:48:31.551796 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:31.551761 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:48:32.656965 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:32.656924 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:48:32.752752 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:32.752714 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:48:33.185230 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:33.185192 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:48:33.286143 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:33.286106 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:48:33.377393 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:33.377355 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-08e484-74f7fb699d-tr96z"] Sep 10 13:48:33.381262 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:33.381243 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-08e484-74f7fb699d-tr96z" Sep 10 13:48:33.383922 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:33.383907 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-08e484\"" Sep 10 13:48:33.384071 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:33.384053 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-08e484-dockercfg-wntm8\"" Sep 10 13:48:33.392533 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:33.392505 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-08e484-74f7fb699d-tr96z"] Sep 10 13:48:33.444228 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:33.444131 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6w87\" (UniqueName: \"kubernetes.io/projected/d4cb3d14-a2aa-4a71-a1d2-61f5b61d9c7d-kube-api-access-h6w87\") pod \"payload-pre-processing-e2e-ipp-cleanup-08e484-74f7fb699d-tr96z\" (UID: \"d4cb3d14-a2aa-4a71-a1d2-61f5b61d9c7d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-08e484-74f7fb699d-tr96z" Sep 10 13:48:33.444228 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:33.444220 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4cb3d14-a2aa-4a71-a1d2-61f5b61d9c7d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-08e484-74f7fb699d-tr96z\" (UID: \"d4cb3d14-a2aa-4a71-a1d2-61f5b61d9c7d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-08e484-74f7fb699d-tr96z" Sep 10 13:48:33.545258 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:33.545223 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6w87\" (UniqueName: \"kubernetes.io/projected/d4cb3d14-a2aa-4a71-a1d2-61f5b61d9c7d-kube-api-access-h6w87\") pod \"payload-pre-processing-e2e-ipp-cleanup-08e484-74f7fb699d-tr96z\" (UID: \"d4cb3d14-a2aa-4a71-a1d2-61f5b61d9c7d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-08e484-74f7fb699d-tr96z" Sep 10 13:48:33.545476 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:33.545300 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4cb3d14-a2aa-4a71-a1d2-61f5b61d9c7d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-08e484-74f7fb699d-tr96z\" (UID: \"d4cb3d14-a2aa-4a71-a1d2-61f5b61d9c7d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-08e484-74f7fb699d-tr96z" Sep 10 13:48:33.546009 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:33.545985 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4cb3d14-a2aa-4a71-a1d2-61f5b61d9c7d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-08e484-74f7fb699d-tr96z\" (UID: \"d4cb3d14-a2aa-4a71-a1d2-61f5b61d9c7d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-08e484-74f7fb699d-tr96z" Sep 10 13:48:33.554243 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:33.554214 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6w87\" (UniqueName: \"kubernetes.io/projected/d4cb3d14-a2aa-4a71-a1d2-61f5b61d9c7d-kube-api-access-h6w87\") pod \"payload-pre-processing-e2e-ipp-cleanup-08e484-74f7fb699d-tr96z\" (UID: \"d4cb3d14-a2aa-4a71-a1d2-61f5b61d9c7d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-08e484-74f7fb699d-tr96z" Sep 10 13:48:33.692636 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:33.692599 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-08e484-74f7fb699d-tr96z" Sep 10 13:48:33.978375 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:33.978346 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-08e484-74f7fb699d-tr96z"] Sep 10 13:48:33.979909 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:48:33.979880 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4cb3d14_a2aa_4a71_a1d2_61f5b61d9c7d.slice/crio-6d083e226a872737aef6249482ea4cd6bdea7b6e3fdc050c1eb5361624b86ce5 WatchSource:0}: Error finding container 6d083e226a872737aef6249482ea4cd6bdea7b6e3fdc050c1eb5361624b86ce5: Status 404 returned error can't find the container with id 6d083e226a872737aef6249482ea4cd6bdea7b6e3fdc050c1eb5361624b86ce5 Sep 10 13:48:34.466990 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:34.466951 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-08e484-74f7fb699d-tr96z" event={"ID":"d4cb3d14-a2aa-4a71-a1d2-61f5b61d9c7d","Type":"ContainerStarted","Data":"e5bdc3037cd26aad38719332a21c7059fd441a242ed9d35b87572eef5232b016"} Sep 10 13:48:34.467187 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:34.467049 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-08e484-74f7fb699d-tr96z" event={"ID":"d4cb3d14-a2aa-4a71-a1d2-61f5b61d9c7d","Type":"ContainerStarted","Data":"6d083e226a872737aef6249482ea4cd6bdea7b6e3fdc050c1eb5361624b86ce5"} Sep 10 13:48:34.467187 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:34.467083 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-08e484-74f7fb699d-tr96z" Sep 10 13:48:34.486304 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:34.486250 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-08e484-74f7fb699d-tr96z" podStartSLOduration=1.486232513 podStartE2EDuration="1.486232513s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:48:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:48:34.484000605 +0000 UTC m=+1824.378718023" watchObservedRunningTime="2026-09-10 13:48:34.486232513 +0000 UTC m=+1824.380949931" Sep 10 13:48:45.477069 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:45.477036 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-08e484-74f7fb699d-tr96z" Sep 10 13:48:51.595772 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:51.595667 2560 generic.go:356] "Generic (PLEG): container finished" podID="37174f02-8469-445e-be70-3bed424acbe2" containerID="ee61dc866367f78bab0058e7913d047016dab9af9afc873cfa2b0b5e70131945" exitCode=0 Sep 10 13:48:51.595772 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:51.595757 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" event={"ID":"37174f02-8469-445e-be70-3bed424acbe2","Type":"ContainerDied","Data":"ee61dc866367f78bab0058e7913d047016dab9af9afc873cfa2b0b5e70131945"} Sep 10 13:48:51.596383 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:51.596365 2560 scope.go:117] "RemoveContainer" containerID="ee61dc866367f78bab0058e7913d047016dab9af9afc873cfa2b0b5e70131945" Sep 10 13:48:51.597510 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:51.597484 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:48:51.597575 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:51.597545 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:48:51.597613 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:51.597573 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"} Sep 10 13:48:52.604486 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:52.604446 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" event={"ID":"37174f02-8469-445e-be70-3bed424acbe2","Type":"ContainerStarted","Data":"ed257d1d833c1f5e30e897fa1442d4f2e3b96d6ee2025f2513e9f104e012cd67"} Sep 10 13:48:52.689278 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:52.689241 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:48:52.689532 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:52.689509 2560 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body= Sep 10 13:48:52.689580 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:52.689559 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" podUID="37174f02-8469-445e-be70-3bed424acbe2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" Sep 10 13:48:53.689076 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:53.689038 2560 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body= Sep 10 13:48:53.689521 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:53.689095 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" podUID="37174f02-8469-445e-be70-3bed424acbe2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" Sep 10 13:48:54.689297 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:54.689259 2560 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body= Sep 10 13:48:54.689728 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:54.689316 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" podUID="37174f02-8469-445e-be70-3bed424acbe2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" Sep 10 13:48:55.688830 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:55.688797 2560 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body= Sep 10 13:48:55.689069 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:55.689047 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" podUID="37174f02-8469-445e-be70-3bed424acbe2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" Sep 10 13:48:56.688827 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:56.688786 2560 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body= Sep 10 13:48:56.689247 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:56.688849 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" podUID="37174f02-8469-445e-be70-3bed424acbe2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" Sep 10 13:48:56.821937 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:56.821904 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:48:57.688788 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:57.688733 2560 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body= Sep 10 13:48:57.689006 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:57.688813 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" podUID="37174f02-8469-445e-be70-3bed424acbe2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" Sep 10 13:48:58.693301 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:58.693269 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:48:58.693745 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:58.693638 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:48:58.694252 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:48:58.694235 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:49:03.219303 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:03.219260 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:49:03.248576 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:03.248541 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-08e484-74f7fb699d-tr96z"] Sep 10 13:49:03.248900 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:03.248870 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-08e484-74f7fb699d-tr96z" podUID="d4cb3d14-a2aa-4a71-a1d2-61f5b61d9c7d" containerName="payload-pre-processing" containerID="cri-o://e5bdc3037cd26aad38719332a21c7059fd441a242ed9d35b87572eef5232b016" gracePeriod=30 Sep 10 13:49:03.492351 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:03.492321 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:49:03.506615 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:03.506586 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-08e484-74f7fb699d-tr96z" Sep 10 13:49:03.555025 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:03.554268 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4cb3d14-a2aa-4a71-a1d2-61f5b61d9c7d-plugins-config-volume\") pod \"d4cb3d14-a2aa-4a71-a1d2-61f5b61d9c7d\" (UID: \"d4cb3d14-a2aa-4a71-a1d2-61f5b61d9c7d\") " Sep 10 13:49:03.555025 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:03.554319 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h6w87\" (UniqueName: \"kubernetes.io/projected/d4cb3d14-a2aa-4a71-a1d2-61f5b61d9c7d-kube-api-access-h6w87\") pod \"d4cb3d14-a2aa-4a71-a1d2-61f5b61d9c7d\" (UID: \"d4cb3d14-a2aa-4a71-a1d2-61f5b61d9c7d\") " Sep 10 13:49:03.555378 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:03.555347 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4cb3d14-a2aa-4a71-a1d2-61f5b61d9c7d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d4cb3d14-a2aa-4a71-a1d2-61f5b61d9c7d" (UID: "d4cb3d14-a2aa-4a71-a1d2-61f5b61d9c7d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:49:03.558443 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:03.558411 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4cb3d14-a2aa-4a71-a1d2-61f5b61d9c7d-kube-api-access-h6w87" (OuterVolumeSpecName: "kube-api-access-h6w87") pod "d4cb3d14-a2aa-4a71-a1d2-61f5b61d9c7d" (UID: "d4cb3d14-a2aa-4a71-a1d2-61f5b61d9c7d"). InnerVolumeSpecName "kube-api-access-h6w87". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:49:03.561149 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:03.561122 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:49:03.655245 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:03.655197 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4cb3d14-a2aa-4a71-a1d2-61f5b61d9c7d-plugins-config-volume\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:49:03.655245 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:03.655236 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6w87\" (UniqueName: \"kubernetes.io/projected/d4cb3d14-a2aa-4a71-a1d2-61f5b61d9c7d-kube-api-access-h6w87\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:49:03.670473 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:03.670397 2560 generic.go:356] "Generic (PLEG): container finished" podID="d4cb3d14-a2aa-4a71-a1d2-61f5b61d9c7d" containerID="e5bdc3037cd26aad38719332a21c7059fd441a242ed9d35b87572eef5232b016" exitCode=0 Sep 10 13:49:03.670626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:03.670477 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-08e484-74f7fb699d-tr96z" Sep 10 13:49:03.670626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:03.670517 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-08e484-74f7fb699d-tr96z" event={"ID":"d4cb3d14-a2aa-4a71-a1d2-61f5b61d9c7d","Type":"ContainerDied","Data":"e5bdc3037cd26aad38719332a21c7059fd441a242ed9d35b87572eef5232b016"} Sep 10 13:49:03.670626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:03.670564 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-08e484-74f7fb699d-tr96z" event={"ID":"d4cb3d14-a2aa-4a71-a1d2-61f5b61d9c7d","Type":"ContainerDied","Data":"6d083e226a872737aef6249482ea4cd6bdea7b6e3fdc050c1eb5361624b86ce5"} Sep 10 13:49:03.670626 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:03.670584 2560 scope.go:117] "RemoveContainer" containerID="e5bdc3037cd26aad38719332a21c7059fd441a242ed9d35b87572eef5232b016" Sep 10 13:49:03.681028 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:03.681002 2560 scope.go:117] "RemoveContainer" containerID="e5bdc3037cd26aad38719332a21c7059fd441a242ed9d35b87572eef5232b016" Sep 10 13:49:03.681320 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:49:03.681303 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5bdc3037cd26aad38719332a21c7059fd441a242ed9d35b87572eef5232b016\": container with ID starting with e5bdc3037cd26aad38719332a21c7059fd441a242ed9d35b87572eef5232b016 not found: ID does not exist" containerID="e5bdc3037cd26aad38719332a21c7059fd441a242ed9d35b87572eef5232b016" Sep 10 13:49:03.681392 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:03.681332 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5bdc3037cd26aad38719332a21c7059fd441a242ed9d35b87572eef5232b016"} err="failed to get container status \"e5bdc3037cd26aad38719332a21c7059fd441a242ed9d35b87572eef5232b016\": rpc error: code = NotFound desc = could not find container \"e5bdc3037cd26aad38719332a21c7059fd441a242ed9d35b87572eef5232b016\": container with ID starting with e5bdc3037cd26aad38719332a21c7059fd441a242ed9d35b87572eef5232b016 not found: ID does not exist" Sep 10 13:49:03.695049 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:03.695013 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-08e484-74f7fb699d-tr96z"] Sep 10 13:49:03.699667 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:03.699629 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-08e484-74f7fb699d-tr96z"] Sep 10 13:49:04.024998 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:04.024959 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:49:04.563902 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:04.563864 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4cb3d14-a2aa-4a71-a1d2-61f5b61d9c7d" path="/var/lib/kubelet/pods/d4cb3d14-a2aa-4a71-a1d2-61f5b61d9c7d/volumes" Sep 10 13:49:10.097057 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:10.097014 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:49:10.178560 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:10.178523 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:49:10.418189 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:10.418160 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj"] Sep 10 13:49:10.418393 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:10.418371 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" podUID="37174f02-8469-445e-be70-3bed424acbe2" containerName="istio-proxy" containerID="cri-o://ed257d1d833c1f5e30e897fa1442d4f2e3b96d6ee2025f2513e9f104e012cd67" gracePeriod=30 Sep 10 13:49:10.472254 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:10.472223 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:49:13.696385 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:13.696343 2560 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Sep 10 13:49:13.696781 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:13.696430 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" podUID="37174f02-8469-445e-be70-3bed424acbe2" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 13:49:14.893719 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:14.893680 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:49:15.666636 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.666606 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:49:15.744063 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.743966 2560 generic.go:356] "Generic (PLEG): container finished" podID="37174f02-8469-445e-be70-3bed424acbe2" containerID="ed257d1d833c1f5e30e897fa1442d4f2e3b96d6ee2025f2513e9f104e012cd67" exitCode=0 Sep 10 13:49:15.744063 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.744035 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" Sep 10 13:49:15.744063 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.744039 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" event={"ID":"37174f02-8469-445e-be70-3bed424acbe2","Type":"ContainerDied","Data":"ed257d1d833c1f5e30e897fa1442d4f2e3b96d6ee2025f2513e9f104e012cd67"} Sep 10 13:49:15.744373 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.744111 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj" event={"ID":"37174f02-8469-445e-be70-3bed424acbe2","Type":"ContainerDied","Data":"5a5e3831c1f304027dd2d5979ea45a7770a7b13d239d3b426c8475794f55add1"} Sep 10 13:49:15.744373 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.744137 2560 scope.go:117] "RemoveContainer" containerID="ed257d1d833c1f5e30e897fa1442d4f2e3b96d6ee2025f2513e9f104e012cd67" Sep 10 13:49:15.757027 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.757001 2560 scope.go:117] "RemoveContainer" containerID="ee61dc866367f78bab0058e7913d047016dab9af9afc873cfa2b0b5e70131945" Sep 10 13:49:15.766326 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.766307 2560 scope.go:117] "RemoveContainer" containerID="ed257d1d833c1f5e30e897fa1442d4f2e3b96d6ee2025f2513e9f104e012cd67" Sep 10 13:49:15.766684 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:49:15.766665 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed257d1d833c1f5e30e897fa1442d4f2e3b96d6ee2025f2513e9f104e012cd67\": container with ID starting with ed257d1d833c1f5e30e897fa1442d4f2e3b96d6ee2025f2513e9f104e012cd67 not found: ID does not exist" containerID="ed257d1d833c1f5e30e897fa1442d4f2e3b96d6ee2025f2513e9f104e012cd67" Sep 10 13:49:15.766730 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.766695 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed257d1d833c1f5e30e897fa1442d4f2e3b96d6ee2025f2513e9f104e012cd67"} err="failed to get container status \"ed257d1d833c1f5e30e897fa1442d4f2e3b96d6ee2025f2513e9f104e012cd67\": rpc error: code = NotFound desc = could not find container \"ed257d1d833c1f5e30e897fa1442d4f2e3b96d6ee2025f2513e9f104e012cd67\": container with ID starting with ed257d1d833c1f5e30e897fa1442d4f2e3b96d6ee2025f2513e9f104e012cd67 not found: ID does not exist" Sep 10 13:49:15.766730 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.766716 2560 scope.go:117] "RemoveContainer" containerID="ee61dc866367f78bab0058e7913d047016dab9af9afc873cfa2b0b5e70131945" Sep 10 13:49:15.766994 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:49:15.766973 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee61dc866367f78bab0058e7913d047016dab9af9afc873cfa2b0b5e70131945\": container with ID starting with ee61dc866367f78bab0058e7913d047016dab9af9afc873cfa2b0b5e70131945 not found: ID does not exist" containerID="ee61dc866367f78bab0058e7913d047016dab9af9afc873cfa2b0b5e70131945" Sep 10 13:49:15.767054 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.766999 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee61dc866367f78bab0058e7913d047016dab9af9afc873cfa2b0b5e70131945"} err="failed to get container status \"ee61dc866367f78bab0058e7913d047016dab9af9afc873cfa2b0b5e70131945\": rpc error: code = NotFound desc = could not find container \"ee61dc866367f78bab0058e7913d047016dab9af9afc873cfa2b0b5e70131945\": container with ID starting with ee61dc866367f78bab0058e7913d047016dab9af9afc873cfa2b0b5e70131945 not found: ID does not exist" Sep 10 13:49:15.770311 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.770291 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37174f02-8469-445e-be70-3bed424acbe2-workload-certs\") pod \"37174f02-8469-445e-be70-3bed424acbe2\" (UID: \"37174f02-8469-445e-be70-3bed424acbe2\") " Sep 10 13:49:15.770393 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.770369 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37174f02-8469-445e-be70-3bed424acbe2-workload-socket\") pod \"37174f02-8469-445e-be70-3bed424acbe2\" (UID: \"37174f02-8469-445e-be70-3bed424acbe2\") " Sep 10 13:49:15.770506 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.770490 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37174f02-8469-445e-be70-3bed424acbe2-credential-socket\") pod \"37174f02-8469-445e-be70-3bed424acbe2\" (UID: \"37174f02-8469-445e-be70-3bed424acbe2\") " Sep 10 13:49:15.770569 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.770535 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37174f02-8469-445e-be70-3bed424acbe2-istio-envoy\") pod \"37174f02-8469-445e-be70-3bed424acbe2\" (UID: \"37174f02-8469-445e-be70-3bed424acbe2\") " Sep 10 13:49:15.770627 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.770566 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37174f02-8469-445e-be70-3bed424acbe2-istio-token\") pod \"37174f02-8469-445e-be70-3bed424acbe2\" (UID: \"37174f02-8469-445e-be70-3bed424acbe2\") " Sep 10 13:49:15.770627 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.770607 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37174f02-8469-445e-be70-3bed424acbe2-istio-data\") pod \"37174f02-8469-445e-be70-3bed424acbe2\" (UID: \"37174f02-8469-445e-be70-3bed424acbe2\") " Sep 10 13:49:15.770734 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.770620 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37174f02-8469-445e-be70-3bed424acbe2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "37174f02-8469-445e-be70-3bed424acbe2" (UID: "37174f02-8469-445e-be70-3bed424acbe2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:49:15.770734 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.770653 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kr962\" (UniqueName: \"kubernetes.io/projected/37174f02-8469-445e-be70-3bed424acbe2-kube-api-access-kr962\") pod \"37174f02-8469-445e-be70-3bed424acbe2\" (UID: \"37174f02-8469-445e-be70-3bed424acbe2\") " Sep 10 13:49:15.770734 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.770681 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37174f02-8469-445e-be70-3bed424acbe2-istiod-ca-cert\") pod \"37174f02-8469-445e-be70-3bed424acbe2\" (UID: \"37174f02-8469-445e-be70-3bed424acbe2\") " Sep 10 13:49:15.770734 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.770723 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37174f02-8469-445e-be70-3bed424acbe2-istio-podinfo\") pod \"37174f02-8469-445e-be70-3bed424acbe2\" (UID: \"37174f02-8469-445e-be70-3bed424acbe2\") " Sep 10 13:49:15.770941 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.770728 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37174f02-8469-445e-be70-3bed424acbe2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "37174f02-8469-445e-be70-3bed424acbe2" (UID: "37174f02-8469-445e-be70-3bed424acbe2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:49:15.770941 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.770740 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37174f02-8469-445e-be70-3bed424acbe2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "37174f02-8469-445e-be70-3bed424acbe2" (UID: "37174f02-8469-445e-be70-3bed424acbe2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:49:15.770941 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.770886 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37174f02-8469-445e-be70-3bed424acbe2-istio-data" (OuterVolumeSpecName: "istio-data") pod "37174f02-8469-445e-be70-3bed424acbe2" (UID: "37174f02-8469-445e-be70-3bed424acbe2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:49:15.771096 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.771060 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37174f02-8469-445e-be70-3bed424acbe2-workload-certs\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:49:15.771096 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.771076 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37174f02-8469-445e-be70-3bed424acbe2-workload-socket\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:49:15.771096 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.771085 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37174f02-8469-445e-be70-3bed424acbe2-credential-socket\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:49:15.771096 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.771081 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37174f02-8469-445e-be70-3bed424acbe2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "37174f02-8469-445e-be70-3bed424acbe2" (UID: "37174f02-8469-445e-be70-3bed424acbe2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:49:15.771096 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.771094 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37174f02-8469-445e-be70-3bed424acbe2-istio-data\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:49:15.772788 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.772759 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/37174f02-8469-445e-be70-3bed424acbe2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "37174f02-8469-445e-be70-3bed424acbe2" (UID: "37174f02-8469-445e-be70-3bed424acbe2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 13:49:15.772788 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.772779 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37174f02-8469-445e-be70-3bed424acbe2-istio-token" (OuterVolumeSpecName: "istio-token") pod "37174f02-8469-445e-be70-3bed424acbe2" (UID: "37174f02-8469-445e-be70-3bed424acbe2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:49:15.773073 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.773053 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37174f02-8469-445e-be70-3bed424acbe2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "37174f02-8469-445e-be70-3bed424acbe2" (UID: "37174f02-8469-445e-be70-3bed424acbe2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:49:15.773134 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.773063 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37174f02-8469-445e-be70-3bed424acbe2-kube-api-access-kr962" (OuterVolumeSpecName: "kube-api-access-kr962") pod "37174f02-8469-445e-be70-3bed424acbe2" (UID: "37174f02-8469-445e-be70-3bed424acbe2"). InnerVolumeSpecName "kube-api-access-kr962". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:49:15.871634 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.871594 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37174f02-8469-445e-be70-3bed424acbe2-istiod-ca-cert\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:49:15.871634 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.871624 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37174f02-8469-445e-be70-3bed424acbe2-istio-podinfo\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:49:15.871634 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.871636 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37174f02-8469-445e-be70-3bed424acbe2-istio-envoy\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:49:15.871874 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.871648 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37174f02-8469-445e-be70-3bed424acbe2-istio-token\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:49:15.871874 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:15.871660 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kr962\" (UniqueName: \"kubernetes.io/projected/37174f02-8469-445e-be70-3bed424acbe2-kube-api-access-kr962\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:49:16.068039 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:16.068001 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj"] Sep 10 13:49:16.072795 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:16.072763 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-08e484-openshift-default-84d87c947d-q9qlj"] Sep 10 13:49:16.563368 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:16.563334 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37174f02-8469-445e-be70-3bed424acbe2" path="/var/lib/kubelet/pods/37174f02-8469-445e-be70-3bed424acbe2/volumes" Sep 10 13:49:30.332514 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:30.332469 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:49:30.370190 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:30.370140 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-41abaf-56d9cc7b59-gmlzl"] Sep 10 13:49:30.370528 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:30.370498 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-41abaf-56d9cc7b59-gmlzl" podUID="796ef08b-1b48-4811-8a48-1aa630675f0e" containerName="payload-processing" containerID="cri-o://90470bf3c972be33b10f4dcb4d0d403bf38a4f40ec301ad8d38bf00f13294ce9" gracePeriod=30 Sep 10 13:49:30.433073 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:30.433017 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:49:30.533681 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:30.533633 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:49:30.600004 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:30.599923 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:49:30.635008 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:30.634977 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-41abaf-56d9cc7b59-gmlzl" Sep 10 13:49:30.710858 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:30.710820 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/796ef08b-1b48-4811-8a48-1aa630675f0e-plugins-config-volume\") pod \"796ef08b-1b48-4811-8a48-1aa630675f0e\" (UID: \"796ef08b-1b48-4811-8a48-1aa630675f0e\") " Sep 10 13:49:30.711185 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:30.711155 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kxnjs\" (UniqueName: \"kubernetes.io/projected/796ef08b-1b48-4811-8a48-1aa630675f0e-kube-api-access-kxnjs\") pod \"796ef08b-1b48-4811-8a48-1aa630675f0e\" (UID: \"796ef08b-1b48-4811-8a48-1aa630675f0e\") " Sep 10 13:49:30.711386 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:30.711337 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/796ef08b-1b48-4811-8a48-1aa630675f0e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "796ef08b-1b48-4811-8a48-1aa630675f0e" (UID: "796ef08b-1b48-4811-8a48-1aa630675f0e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:49:30.711691 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:30.711668 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/796ef08b-1b48-4811-8a48-1aa630675f0e-plugins-config-volume\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:49:30.714153 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:30.714125 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/796ef08b-1b48-4811-8a48-1aa630675f0e-kube-api-access-kxnjs" (OuterVolumeSpecName: "kube-api-access-kxnjs") pod "796ef08b-1b48-4811-8a48-1aa630675f0e" (UID: "796ef08b-1b48-4811-8a48-1aa630675f0e"). InnerVolumeSpecName "kube-api-access-kxnjs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:49:30.812660 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:30.812607 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxnjs\" (UniqueName: \"kubernetes.io/projected/796ef08b-1b48-4811-8a48-1aa630675f0e-kube-api-access-kxnjs\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:49:30.836192 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:30.836158 2560 generic.go:356] "Generic (PLEG): container finished" podID="796ef08b-1b48-4811-8a48-1aa630675f0e" containerID="90470bf3c972be33b10f4dcb4d0d403bf38a4f40ec301ad8d38bf00f13294ce9" exitCode=0 Sep 10 13:49:30.836340 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:30.836210 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-41abaf-56d9cc7b59-gmlzl" Sep 10 13:49:30.836340 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:30.836263 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-41abaf-56d9cc7b59-gmlzl" event={"ID":"796ef08b-1b48-4811-8a48-1aa630675f0e","Type":"ContainerDied","Data":"90470bf3c972be33b10f4dcb4d0d403bf38a4f40ec301ad8d38bf00f13294ce9"} Sep 10 13:49:30.836340 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:30.836315 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-41abaf-56d9cc7b59-gmlzl" event={"ID":"796ef08b-1b48-4811-8a48-1aa630675f0e","Type":"ContainerDied","Data":"2c9a614205100cee55b00a7fd4aba3294b158d72413571a4e70be3f7f5832feb"} Sep 10 13:49:30.836340 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:30.836336 2560 scope.go:117] "RemoveContainer" containerID="90470bf3c972be33b10f4dcb4d0d403bf38a4f40ec301ad8d38bf00f13294ce9" Sep 10 13:49:30.846150 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:30.846131 2560 scope.go:117] "RemoveContainer" containerID="90470bf3c972be33b10f4dcb4d0d403bf38a4f40ec301ad8d38bf00f13294ce9" Sep 10 13:49:30.846427 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:49:30.846386 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90470bf3c972be33b10f4dcb4d0d403bf38a4f40ec301ad8d38bf00f13294ce9\": container with ID starting with 90470bf3c972be33b10f4dcb4d0d403bf38a4f40ec301ad8d38bf00f13294ce9 not found: ID does not exist" containerID="90470bf3c972be33b10f4dcb4d0d403bf38a4f40ec301ad8d38bf00f13294ce9" Sep 10 13:49:30.846489 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:30.846425 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90470bf3c972be33b10f4dcb4d0d403bf38a4f40ec301ad8d38bf00f13294ce9"} err="failed to get container status \"90470bf3c972be33b10f4dcb4d0d403bf38a4f40ec301ad8d38bf00f13294ce9\": rpc error: code = NotFound desc = could not find container \"90470bf3c972be33b10f4dcb4d0d403bf38a4f40ec301ad8d38bf00f13294ce9\": container with ID starting with 90470bf3c972be33b10f4dcb4d0d403bf38a4f40ec301ad8d38bf00f13294ce9 not found: ID does not exist" Sep 10 13:49:30.859578 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:30.859515 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-41abaf-56d9cc7b59-gmlzl"] Sep 10 13:49:30.863683 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:30.863657 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-41abaf-56d9cc7b59-gmlzl"] Sep 10 13:49:30.948497 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:30.948462 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:49:32.564513 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:32.564477 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="796ef08b-1b48-4811-8a48-1aa630675f0e" path="/var/lib/kubelet/pods/796ef08b-1b48-4811-8a48-1aa630675f0e/volumes" Sep 10 13:49:36.020313 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:36.020267 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:49:47.584392 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:49:47.584352 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:50:13.125478 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:13.125444 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:50:13.223806 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:13.223767 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:50:17.295875 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:17.295841 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:50:19.212304 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:19.212251 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:50:19.360645 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:19.360610 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:50:19.531266 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:19.531178 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:50:21.344569 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:21.344527 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:50:30.532706 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:30.532667 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:50:30.566523 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:30.566490 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-645758-d9d5fcd4b-pxsz5"] Sep 10 13:50:30.566827 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:30.566673 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-645758-d9d5fcd4b-pxsz5" podUID="17221ec2-5ef2-4ef3-9889-32eb821aff1d" containerName="payload-pre-processing" containerID="cri-o://92148123fd228359cde5da8fc8e24e5a1abeb04f3ae7726e55cfc0b37d5a2eee" gracePeriod=30 Sep 10 13:50:30.632249 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:30.632213 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:50:30.732095 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:30.732058 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:50:30.833664 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:30.833623 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:50:30.833837 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:30.833803 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-645758-d9d5fcd4b-pxsz5" Sep 10 13:50:30.884650 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:30.884616 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7tbh\" (UniqueName: \"kubernetes.io/projected/17221ec2-5ef2-4ef3-9889-32eb821aff1d-kube-api-access-x7tbh\") pod \"17221ec2-5ef2-4ef3-9889-32eb821aff1d\" (UID: \"17221ec2-5ef2-4ef3-9889-32eb821aff1d\") " Sep 10 13:50:30.884844 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:30.884680 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17221ec2-5ef2-4ef3-9889-32eb821aff1d-plugins-config-volume\") pod \"17221ec2-5ef2-4ef3-9889-32eb821aff1d\" (UID: \"17221ec2-5ef2-4ef3-9889-32eb821aff1d\") " Sep 10 13:50:30.885134 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:30.885105 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17221ec2-5ef2-4ef3-9889-32eb821aff1d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "17221ec2-5ef2-4ef3-9889-32eb821aff1d" (UID: "17221ec2-5ef2-4ef3-9889-32eb821aff1d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:50:30.886678 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:30.886654 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17221ec2-5ef2-4ef3-9889-32eb821aff1d-kube-api-access-x7tbh" (OuterVolumeSpecName: "kube-api-access-x7tbh") pod "17221ec2-5ef2-4ef3-9889-32eb821aff1d" (UID: "17221ec2-5ef2-4ef3-9889-32eb821aff1d"). InnerVolumeSpecName "kube-api-access-x7tbh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:50:30.986087 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:30.986050 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17221ec2-5ef2-4ef3-9889-32eb821aff1d-plugins-config-volume\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:50:30.986087 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:30.986080 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7tbh\" (UniqueName: \"kubernetes.io/projected/17221ec2-5ef2-4ef3-9889-32eb821aff1d-kube-api-access-x7tbh\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:50:31.146836 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:31.146737 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:50:31.210744 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:31.210708 2560 generic.go:356] "Generic (PLEG): container finished" podID="17221ec2-5ef2-4ef3-9889-32eb821aff1d" containerID="92148123fd228359cde5da8fc8e24e5a1abeb04f3ae7726e55cfc0b37d5a2eee" exitCode=0 Sep 10 13:50:31.210935 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:31.210765 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-645758-d9d5fcd4b-pxsz5" Sep 10 13:50:31.210935 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:31.210789 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-645758-d9d5fcd4b-pxsz5" event={"ID":"17221ec2-5ef2-4ef3-9889-32eb821aff1d","Type":"ContainerDied","Data":"92148123fd228359cde5da8fc8e24e5a1abeb04f3ae7726e55cfc0b37d5a2eee"} Sep 10 13:50:31.210935 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:31.210848 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-645758-d9d5fcd4b-pxsz5" event={"ID":"17221ec2-5ef2-4ef3-9889-32eb821aff1d","Type":"ContainerDied","Data":"12739fab42d17594bf9be869a1a7e4d55b6d320f1c09676e72fb79018d57575d"} Sep 10 13:50:31.210935 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:31.210870 2560 scope.go:117] "RemoveContainer" containerID="92148123fd228359cde5da8fc8e24e5a1abeb04f3ae7726e55cfc0b37d5a2eee" Sep 10 13:50:31.220360 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:31.220343 2560 scope.go:117] "RemoveContainer" containerID="92148123fd228359cde5da8fc8e24e5a1abeb04f3ae7726e55cfc0b37d5a2eee" Sep 10 13:50:31.220677 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:50:31.220658 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92148123fd228359cde5da8fc8e24e5a1abeb04f3ae7726e55cfc0b37d5a2eee\": container with ID starting with 92148123fd228359cde5da8fc8e24e5a1abeb04f3ae7726e55cfc0b37d5a2eee not found: ID does not exist" containerID="92148123fd228359cde5da8fc8e24e5a1abeb04f3ae7726e55cfc0b37d5a2eee" Sep 10 13:50:31.220739 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:31.220685 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92148123fd228359cde5da8fc8e24e5a1abeb04f3ae7726e55cfc0b37d5a2eee"} err="failed to get container status \"92148123fd228359cde5da8fc8e24e5a1abeb04f3ae7726e55cfc0b37d5a2eee\": rpc error: code = NotFound desc = could not find container \"92148123fd228359cde5da8fc8e24e5a1abeb04f3ae7726e55cfc0b37d5a2eee\": container with ID starting with 92148123fd228359cde5da8fc8e24e5a1abeb04f3ae7726e55cfc0b37d5a2eee not found: ID does not exist" Sep 10 13:50:31.240152 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:31.240113 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-645758-d9d5fcd4b-pxsz5"] Sep 10 13:50:31.242703 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:31.242684 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-645758-d9d5fcd4b-pxsz5"] Sep 10 13:50:32.563012 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:32.562980 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17221ec2-5ef2-4ef3-9889-32eb821aff1d" path="/var/lib/kubelet/pods/17221ec2-5ef2-4ef3-9889-32eb821aff1d/volumes" Sep 10 13:50:37.314519 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:37.314482 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:50:37.477459 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:37.477425 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:50:37.621956 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:37.621860 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:50:38.175894 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:38.175859 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:50:48.585283 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:48.585251 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:50:48.611744 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:48.611707 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-767dce-6db9bbd447-j75c5"] Sep 10 13:50:48.611961 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:48.611930 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-767dce-6db9bbd447-j75c5" podUID="fb9cd903-e9a3-4af1-b443-da43b25180ab" containerName="payload-processing" containerID="cri-o://b472aec8a204239260be1909a3f7ba878777b5ce7f7e6f575725fb3c1d31e7ff" gracePeriod=30 Sep 10 13:50:48.619295 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:48.619263 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-767dce-56ffd89574-cptzg"] Sep 10 13:50:48.619563 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:48.619514 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-767dce-56ffd89574-cptzg" podUID="8374fc76-574f-48bd-aaf8-021f1e4e5b43" containerName="payload-pre-processing" containerID="cri-o://fc5e2a2efa77f7ca7dc8daac5cbaaaf3e3f42967aaf4699d16eff6bb65284b8a" gracePeriod=30 Sep 10 13:50:48.665916 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:48.665885 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:50:48.735747 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:48.735719 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:50:48.812884 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:48.812849 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:50:48.897118 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:48.897091 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-767dce-6db9bbd447-j75c5" Sep 10 13:50:48.899819 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:48.899799 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-767dce-56ffd89574-cptzg" Sep 10 13:50:48.951881 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:48.951833 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8374fc76-574f-48bd-aaf8-021f1e4e5b43-plugins-config-volume\") pod \"8374fc76-574f-48bd-aaf8-021f1e4e5b43\" (UID: \"8374fc76-574f-48bd-aaf8-021f1e4e5b43\") " Sep 10 13:50:48.952071 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:48.951933 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb9cd903-e9a3-4af1-b443-da43b25180ab-plugins-config-volume\") pod \"fb9cd903-e9a3-4af1-b443-da43b25180ab\" (UID: \"fb9cd903-e9a3-4af1-b443-da43b25180ab\") " Sep 10 13:50:48.952071 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:48.951995 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5vxjl\" (UniqueName: \"kubernetes.io/projected/fb9cd903-e9a3-4af1-b443-da43b25180ab-kube-api-access-5vxjl\") pod \"fb9cd903-e9a3-4af1-b443-da43b25180ab\" (UID: \"fb9cd903-e9a3-4af1-b443-da43b25180ab\") " Sep 10 13:50:48.952071 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:48.952032 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hh6v\" (UniqueName: \"kubernetes.io/projected/8374fc76-574f-48bd-aaf8-021f1e4e5b43-kube-api-access-2hh6v\") pod \"8374fc76-574f-48bd-aaf8-021f1e4e5b43\" (UID: \"8374fc76-574f-48bd-aaf8-021f1e4e5b43\") " Sep 10 13:50:48.952314 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:48.952266 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8374fc76-574f-48bd-aaf8-021f1e4e5b43-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8374fc76-574f-48bd-aaf8-021f1e4e5b43" (UID: "8374fc76-574f-48bd-aaf8-021f1e4e5b43"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:50:48.952314 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:48.952294 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb9cd903-e9a3-4af1-b443-da43b25180ab-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fb9cd903-e9a3-4af1-b443-da43b25180ab" (UID: "fb9cd903-e9a3-4af1-b443-da43b25180ab"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:50:48.954898 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:48.954867 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb9cd903-e9a3-4af1-b443-da43b25180ab-kube-api-access-5vxjl" (OuterVolumeSpecName: "kube-api-access-5vxjl") pod "fb9cd903-e9a3-4af1-b443-da43b25180ab" (UID: "fb9cd903-e9a3-4af1-b443-da43b25180ab"). InnerVolumeSpecName "kube-api-access-5vxjl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:50:48.954898 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:48.954882 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8374fc76-574f-48bd-aaf8-021f1e4e5b43-kube-api-access-2hh6v" (OuterVolumeSpecName: "kube-api-access-2hh6v") pod "8374fc76-574f-48bd-aaf8-021f1e4e5b43" (UID: "8374fc76-574f-48bd-aaf8-021f1e4e5b43"). InnerVolumeSpecName "kube-api-access-2hh6v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:50:49.052876 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:49.052832 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hh6v\" (UniqueName: \"kubernetes.io/projected/8374fc76-574f-48bd-aaf8-021f1e4e5b43-kube-api-access-2hh6v\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:50:49.052876 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:49.052869 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8374fc76-574f-48bd-aaf8-021f1e4e5b43-plugins-config-volume\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:50:49.052876 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:49.052878 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb9cd903-e9a3-4af1-b443-da43b25180ab-plugins-config-volume\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:50:49.052876 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:49.052889 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vxjl\" (UniqueName: \"kubernetes.io/projected/fb9cd903-e9a3-4af1-b443-da43b25180ab-kube-api-access-5vxjl\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:50:49.219227 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:49.219196 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:50:49.328711 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:49.328673 2560 generic.go:356] "Generic (PLEG): container finished" podID="8374fc76-574f-48bd-aaf8-021f1e4e5b43" containerID="fc5e2a2efa77f7ca7dc8daac5cbaaaf3e3f42967aaf4699d16eff6bb65284b8a" exitCode=0 Sep 10 13:50:49.328896 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:49.328725 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-767dce-56ffd89574-cptzg" Sep 10 13:50:49.328896 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:49.328755 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-767dce-56ffd89574-cptzg" event={"ID":"8374fc76-574f-48bd-aaf8-021f1e4e5b43","Type":"ContainerDied","Data":"fc5e2a2efa77f7ca7dc8daac5cbaaaf3e3f42967aaf4699d16eff6bb65284b8a"} Sep 10 13:50:49.328896 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:49.328808 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-767dce-56ffd89574-cptzg" event={"ID":"8374fc76-574f-48bd-aaf8-021f1e4e5b43","Type":"ContainerDied","Data":"3ba2dbcb502163d379539248ce8443605bf2836037086168b3054078387218fb"} Sep 10 13:50:49.328896 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:49.328839 2560 scope.go:117] "RemoveContainer" containerID="fc5e2a2efa77f7ca7dc8daac5cbaaaf3e3f42967aaf4699d16eff6bb65284b8a" Sep 10 13:50:49.330285 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:49.330259 2560 generic.go:356] "Generic (PLEG): container finished" podID="fb9cd903-e9a3-4af1-b443-da43b25180ab" containerID="b472aec8a204239260be1909a3f7ba878777b5ce7f7e6f575725fb3c1d31e7ff" exitCode=0 Sep 10 13:50:49.330434 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:49.330289 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-767dce-6db9bbd447-j75c5" event={"ID":"fb9cd903-e9a3-4af1-b443-da43b25180ab","Type":"ContainerDied","Data":"b472aec8a204239260be1909a3f7ba878777b5ce7f7e6f575725fb3c1d31e7ff"} Sep 10 13:50:49.330434 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:49.330308 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-767dce-6db9bbd447-j75c5" Sep 10 13:50:49.330434 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:49.330347 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-767dce-6db9bbd447-j75c5" event={"ID":"fb9cd903-e9a3-4af1-b443-da43b25180ab","Type":"ContainerDied","Data":"f9ff30b2f9ac4ceecc292095c9717be39152b04c83a9d84a020706cd0d403dca"} Sep 10 13:50:49.338753 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:49.338729 2560 scope.go:117] "RemoveContainer" containerID="fc5e2a2efa77f7ca7dc8daac5cbaaaf3e3f42967aaf4699d16eff6bb65284b8a" Sep 10 13:50:49.339078 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:50:49.339051 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc5e2a2efa77f7ca7dc8daac5cbaaaf3e3f42967aaf4699d16eff6bb65284b8a\": container with ID starting with fc5e2a2efa77f7ca7dc8daac5cbaaaf3e3f42967aaf4699d16eff6bb65284b8a not found: ID does not exist" containerID="fc5e2a2efa77f7ca7dc8daac5cbaaaf3e3f42967aaf4699d16eff6bb65284b8a" Sep 10 13:50:49.339178 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:49.339090 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc5e2a2efa77f7ca7dc8daac5cbaaaf3e3f42967aaf4699d16eff6bb65284b8a"} err="failed to get container status \"fc5e2a2efa77f7ca7dc8daac5cbaaaf3e3f42967aaf4699d16eff6bb65284b8a\": rpc error: code = NotFound desc = could not find container \"fc5e2a2efa77f7ca7dc8daac5cbaaaf3e3f42967aaf4699d16eff6bb65284b8a\": container with ID starting with fc5e2a2efa77f7ca7dc8daac5cbaaaf3e3f42967aaf4699d16eff6bb65284b8a not found: ID does not exist" Sep 10 13:50:49.339178 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:49.339114 2560 scope.go:117] "RemoveContainer" containerID="b472aec8a204239260be1909a3f7ba878777b5ce7f7e6f575725fb3c1d31e7ff" Sep 10 13:50:49.348128 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:49.348108 2560 scope.go:117] "RemoveContainer" containerID="b472aec8a204239260be1909a3f7ba878777b5ce7f7e6f575725fb3c1d31e7ff" Sep 10 13:50:49.348420 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:50:49.348389 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b472aec8a204239260be1909a3f7ba878777b5ce7f7e6f575725fb3c1d31e7ff\": container with ID starting with b472aec8a204239260be1909a3f7ba878777b5ce7f7e6f575725fb3c1d31e7ff not found: ID does not exist" containerID="b472aec8a204239260be1909a3f7ba878777b5ce7f7e6f575725fb3c1d31e7ff" Sep 10 13:50:49.348469 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:49.348430 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b472aec8a204239260be1909a3f7ba878777b5ce7f7e6f575725fb3c1d31e7ff"} err="failed to get container status \"b472aec8a204239260be1909a3f7ba878777b5ce7f7e6f575725fb3c1d31e7ff\": rpc error: code = NotFound desc = could not find container \"b472aec8a204239260be1909a3f7ba878777b5ce7f7e6f575725fb3c1d31e7ff\": container with ID starting with b472aec8a204239260be1909a3f7ba878777b5ce7f7e6f575725fb3c1d31e7ff not found: ID does not exist" Sep 10 13:50:49.357893 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:49.357866 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-767dce-6db9bbd447-j75c5"] Sep 10 13:50:49.362221 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:49.362195 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-767dce-6db9bbd447-j75c5"] Sep 10 13:50:49.374850 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:49.374820 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-767dce-56ffd89574-cptzg"] Sep 10 13:50:49.379696 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:49.379667 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-767dce-56ffd89574-cptzg"] Sep 10 13:50:50.563052 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:50.563017 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8374fc76-574f-48bd-aaf8-021f1e4e5b43" path="/var/lib/kubelet/pods/8374fc76-574f-48bd-aaf8-021f1e4e5b43/volumes" Sep 10 13:50:50.563434 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:50.563394 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb9cd903-e9a3-4af1-b443-da43b25180ab" path="/var/lib/kubelet/pods/fb9cd903-e9a3-4af1-b443-da43b25180ab/volumes" Sep 10 13:50:54.737654 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:54.737617 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:50:54.810828 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:54.810795 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:50:55.435718 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:55.435671 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:50:55.508076 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:55.508042 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:50:55.693952 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:55.693868 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm"] Sep 10 13:50:55.694169 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:55.694143 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" podUID="38e4531c-cd84-4583-8cc8-e86e966c398e" containerName="istio-proxy" containerID="cri-o://d26665bb15f4853cd187feb1e0d13d5ea4788a59e2f42214d10f450b09e86a80" gracePeriod=30 Sep 10 13:50:55.718519 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:55.718482 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:50:56.195654 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:50:56.195623 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:51:00.944276 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:00.944245 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:51:01.069186 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.069096 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38e4531c-cd84-4583-8cc8-e86e966c398e-workload-certs\") pod \"38e4531c-cd84-4583-8cc8-e86e966c398e\" (UID: \"38e4531c-cd84-4583-8cc8-e86e966c398e\") " Sep 10 13:51:01.069186 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.069148 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38e4531c-cd84-4583-8cc8-e86e966c398e-istio-data\") pod \"38e4531c-cd84-4583-8cc8-e86e966c398e\" (UID: \"38e4531c-cd84-4583-8cc8-e86e966c398e\") " Sep 10 13:51:01.069444 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.069208 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38e4531c-cd84-4583-8cc8-e86e966c398e-istiod-ca-cert\") pod \"38e4531c-cd84-4583-8cc8-e86e966c398e\" (UID: \"38e4531c-cd84-4583-8cc8-e86e966c398e\") " Sep 10 13:51:01.069444 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.069258 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jrwvw\" (UniqueName: \"kubernetes.io/projected/38e4531c-cd84-4583-8cc8-e86e966c398e-kube-api-access-jrwvw\") pod \"38e4531c-cd84-4583-8cc8-e86e966c398e\" (UID: \"38e4531c-cd84-4583-8cc8-e86e966c398e\") " Sep 10 13:51:01.069444 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.069318 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38e4531c-cd84-4583-8cc8-e86e966c398e-istio-podinfo\") pod \"38e4531c-cd84-4583-8cc8-e86e966c398e\" (UID: \"38e4531c-cd84-4583-8cc8-e86e966c398e\") " Sep 10 13:51:01.069444 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.069343 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38e4531c-cd84-4583-8cc8-e86e966c398e-istio-token\") pod \"38e4531c-cd84-4583-8cc8-e86e966c398e\" (UID: \"38e4531c-cd84-4583-8cc8-e86e966c398e\") " Sep 10 13:51:01.069444 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.069392 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38e4531c-cd84-4583-8cc8-e86e966c398e-istio-envoy\") pod \"38e4531c-cd84-4583-8cc8-e86e966c398e\" (UID: \"38e4531c-cd84-4583-8cc8-e86e966c398e\") " Sep 10 13:51:01.069720 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.069448 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38e4531c-cd84-4583-8cc8-e86e966c398e-workload-socket\") pod \"38e4531c-cd84-4583-8cc8-e86e966c398e\" (UID: \"38e4531c-cd84-4583-8cc8-e86e966c398e\") " Sep 10 13:51:01.069720 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.069472 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38e4531c-cd84-4583-8cc8-e86e966c398e-credential-socket\") pod \"38e4531c-cd84-4583-8cc8-e86e966c398e\" (UID: \"38e4531c-cd84-4583-8cc8-e86e966c398e\") " Sep 10 13:51:01.069720 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.069393 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38e4531c-cd84-4583-8cc8-e86e966c398e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "38e4531c-cd84-4583-8cc8-e86e966c398e" (UID: "38e4531c-cd84-4583-8cc8-e86e966c398e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:51:01.069720 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.069686 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38e4531c-cd84-4583-8cc8-e86e966c398e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "38e4531c-cd84-4583-8cc8-e86e966c398e" (UID: "38e4531c-cd84-4583-8cc8-e86e966c398e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:51:01.069720 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.069461 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38e4531c-cd84-4583-8cc8-e86e966c398e-istio-data" (OuterVolumeSpecName: "istio-data") pod "38e4531c-cd84-4583-8cc8-e86e966c398e" (UID: "38e4531c-cd84-4583-8cc8-e86e966c398e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:51:01.069975 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.069902 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38e4531c-cd84-4583-8cc8-e86e966c398e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "38e4531c-cd84-4583-8cc8-e86e966c398e" (UID: "38e4531c-cd84-4583-8cc8-e86e966c398e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:51:01.070062 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.070041 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38e4531c-cd84-4583-8cc8-e86e966c398e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "38e4531c-cd84-4583-8cc8-e86e966c398e" (UID: "38e4531c-cd84-4583-8cc8-e86e966c398e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:51:01.070439 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.070419 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38e4531c-cd84-4583-8cc8-e86e966c398e-workload-socket\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:51:01.070551 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.070441 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38e4531c-cd84-4583-8cc8-e86e966c398e-credential-socket\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:51:01.070551 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.070456 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38e4531c-cd84-4583-8cc8-e86e966c398e-workload-certs\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:51:01.070551 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.070470 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38e4531c-cd84-4583-8cc8-e86e966c398e-istio-data\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:51:01.070551 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.070483 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38e4531c-cd84-4583-8cc8-e86e966c398e-istiod-ca-cert\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:51:01.072020 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.071994 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38e4531c-cd84-4583-8cc8-e86e966c398e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "38e4531c-cd84-4583-8cc8-e86e966c398e" (UID: "38e4531c-cd84-4583-8cc8-e86e966c398e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:51:01.072137 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.072023 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38e4531c-cd84-4583-8cc8-e86e966c398e-kube-api-access-jrwvw" (OuterVolumeSpecName: "kube-api-access-jrwvw") pod "38e4531c-cd84-4583-8cc8-e86e966c398e" (UID: "38e4531c-cd84-4583-8cc8-e86e966c398e"). InnerVolumeSpecName "kube-api-access-jrwvw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:51:01.072202 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.072157 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38e4531c-cd84-4583-8cc8-e86e966c398e-istio-token" (OuterVolumeSpecName: "istio-token") pod "38e4531c-cd84-4583-8cc8-e86e966c398e" (UID: "38e4531c-cd84-4583-8cc8-e86e966c398e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:51:01.072288 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.072260 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/38e4531c-cd84-4583-8cc8-e86e966c398e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "38e4531c-cd84-4583-8cc8-e86e966c398e" (UID: "38e4531c-cd84-4583-8cc8-e86e966c398e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 13:51:01.172051 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.171996 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38e4531c-cd84-4583-8cc8-e86e966c398e-istio-podinfo\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:51:01.172051 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.172047 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38e4531c-cd84-4583-8cc8-e86e966c398e-istio-token\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:51:01.172051 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.172061 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38e4531c-cd84-4583-8cc8-e86e966c398e-istio-envoy\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:51:01.172311 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.172075 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrwvw\" (UniqueName: \"kubernetes.io/projected/38e4531c-cd84-4583-8cc8-e86e966c398e-kube-api-access-jrwvw\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:51:01.406260 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.406157 2560 generic.go:356] "Generic (PLEG): container finished" podID="38e4531c-cd84-4583-8cc8-e86e966c398e" containerID="d26665bb15f4853cd187feb1e0d13d5ea4788a59e2f42214d10f450b09e86a80" exitCode=0 Sep 10 13:51:01.406260 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.406205 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" Sep 10 13:51:01.406260 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.406249 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" event={"ID":"38e4531c-cd84-4583-8cc8-e86e966c398e","Type":"ContainerDied","Data":"d26665bb15f4853cd187feb1e0d13d5ea4788a59e2f42214d10f450b09e86a80"} Sep 10 13:51:01.406556 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.406303 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm" event={"ID":"38e4531c-cd84-4583-8cc8-e86e966c398e","Type":"ContainerDied","Data":"fd2c68fab4bb0dd1ba3337019df908b10cf07efce43ae4cfe18947d5c3c73f1d"} Sep 10 13:51:01.406556 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.406329 2560 scope.go:117] "RemoveContainer" containerID="d26665bb15f4853cd187feb1e0d13d5ea4788a59e2f42214d10f450b09e86a80" Sep 10 13:51:01.418195 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.418174 2560 scope.go:117] "RemoveContainer" containerID="ec93289e8ef5174fba424e190e88d387b06e6efc0f9b221630238ab2e45fb357" Sep 10 13:51:01.426586 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.426569 2560 scope.go:117] "RemoveContainer" containerID="d26665bb15f4853cd187feb1e0d13d5ea4788a59e2f42214d10f450b09e86a80" Sep 10 13:51:01.426857 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:51:01.426839 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d26665bb15f4853cd187feb1e0d13d5ea4788a59e2f42214d10f450b09e86a80\": container with ID starting with d26665bb15f4853cd187feb1e0d13d5ea4788a59e2f42214d10f450b09e86a80 not found: ID does not exist" containerID="d26665bb15f4853cd187feb1e0d13d5ea4788a59e2f42214d10f450b09e86a80" Sep 10 13:51:01.426935 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.426871 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d26665bb15f4853cd187feb1e0d13d5ea4788a59e2f42214d10f450b09e86a80"} err="failed to get container status \"d26665bb15f4853cd187feb1e0d13d5ea4788a59e2f42214d10f450b09e86a80\": rpc error: code = NotFound desc = could not find container \"d26665bb15f4853cd187feb1e0d13d5ea4788a59e2f42214d10f450b09e86a80\": container with ID starting with d26665bb15f4853cd187feb1e0d13d5ea4788a59e2f42214d10f450b09e86a80 not found: ID does not exist" Sep 10 13:51:01.426935 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.426897 2560 scope.go:117] "RemoveContainer" containerID="ec93289e8ef5174fba424e190e88d387b06e6efc0f9b221630238ab2e45fb357" Sep 10 13:51:01.427156 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:51:01.427139 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec93289e8ef5174fba424e190e88d387b06e6efc0f9b221630238ab2e45fb357\": container with ID starting with ec93289e8ef5174fba424e190e88d387b06e6efc0f9b221630238ab2e45fb357 not found: ID does not exist" containerID="ec93289e8ef5174fba424e190e88d387b06e6efc0f9b221630238ab2e45fb357" Sep 10 13:51:01.427202 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.427161 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec93289e8ef5174fba424e190e88d387b06e6efc0f9b221630238ab2e45fb357"} err="failed to get container status \"ec93289e8ef5174fba424e190e88d387b06e6efc0f9b221630238ab2e45fb357\": rpc error: code = NotFound desc = could not find container \"ec93289e8ef5174fba424e190e88d387b06e6efc0f9b221630238ab2e45fb357\": container with ID starting with ec93289e8ef5174fba424e190e88d387b06e6efc0f9b221630238ab2e45fb357 not found: ID does not exist" Sep 10 13:51:01.434130 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.434105 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm"] Sep 10 13:51:01.438752 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:01.438730 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-767dce-openshift-default-5dd9969884-9hfdm"] Sep 10 13:51:02.563122 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:02.563084 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38e4531c-cd84-4583-8cc8-e86e966c398e" path="/var/lib/kubelet/pods/38e4531c-cd84-4583-8cc8-e86e966c398e/volumes" Sep 10 13:51:06.583068 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:06.583027 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:51:06.614284 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:06.614249 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-89be92-98b7b59cf-9krl9"] Sep 10 13:51:06.614728 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:06.614690 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-89be92-98b7b59cf-9krl9" podUID="d11b889b-57c3-4ae5-8cce-f337308aaa6d" containerName="payload-processing" containerID="cri-o://77e1000b202d967f1a70c8578c8958fadb70c6d0b3a4b607df96d248759ba335" gracePeriod=30 Sep 10 13:51:06.625843 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:06.625536 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-89be92-6f87d675d6-jldxc"] Sep 10 13:51:06.625843 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:06.625745 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-89be92-6f87d675d6-jldxc" podUID="be8293d4-b9ef-4d0f-9686-ae65c483f0f4" containerName="payload-pre-processing" containerID="cri-o://c11c936fe3ecad3e51d1fbae5b7f87fac64267917d897e7e5875b1f55849af86" gracePeriod=30 Sep 10 13:51:06.663684 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:06.663647 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:51:06.736009 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:06.735970 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:51:06.798006 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:06.797968 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:51:06.903476 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:06.903449 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-89be92-98b7b59cf-9krl9" Sep 10 13:51:06.907067 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:06.907044 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-89be92-6f87d675d6-jldxc" Sep 10 13:51:07.028470 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:07.028429 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d11b889b-57c3-4ae5-8cce-f337308aaa6d-plugins-config-volume\") pod \"d11b889b-57c3-4ae5-8cce-f337308aaa6d\" (UID: \"d11b889b-57c3-4ae5-8cce-f337308aaa6d\") " Sep 10 13:51:07.028470 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:07.028476 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/be8293d4-b9ef-4d0f-9686-ae65c483f0f4-plugins-config-volume\") pod \"be8293d4-b9ef-4d0f-9686-ae65c483f0f4\" (UID: \"be8293d4-b9ef-4d0f-9686-ae65c483f0f4\") " Sep 10 13:51:07.028685 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:07.028504 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tplhn\" (UniqueName: \"kubernetes.io/projected/d11b889b-57c3-4ae5-8cce-f337308aaa6d-kube-api-access-tplhn\") pod \"d11b889b-57c3-4ae5-8cce-f337308aaa6d\" (UID: \"d11b889b-57c3-4ae5-8cce-f337308aaa6d\") " Sep 10 13:51:07.028685 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:07.028531 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bgm9l\" (UniqueName: \"kubernetes.io/projected/be8293d4-b9ef-4d0f-9686-ae65c483f0f4-kube-api-access-bgm9l\") pod \"be8293d4-b9ef-4d0f-9686-ae65c483f0f4\" (UID: \"be8293d4-b9ef-4d0f-9686-ae65c483f0f4\") " Sep 10 13:51:07.028843 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:07.028818 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d11b889b-57c3-4ae5-8cce-f337308aaa6d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d11b889b-57c3-4ae5-8cce-f337308aaa6d" (UID: "d11b889b-57c3-4ae5-8cce-f337308aaa6d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:51:07.028917 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:07.028893 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be8293d4-b9ef-4d0f-9686-ae65c483f0f4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "be8293d4-b9ef-4d0f-9686-ae65c483f0f4" (UID: "be8293d4-b9ef-4d0f-9686-ae65c483f0f4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:51:07.030702 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:07.030683 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d11b889b-57c3-4ae5-8cce-f337308aaa6d-kube-api-access-tplhn" (OuterVolumeSpecName: "kube-api-access-tplhn") pod "d11b889b-57c3-4ae5-8cce-f337308aaa6d" (UID: "d11b889b-57c3-4ae5-8cce-f337308aaa6d"). InnerVolumeSpecName "kube-api-access-tplhn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:51:07.030761 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:07.030728 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be8293d4-b9ef-4d0f-9686-ae65c483f0f4-kube-api-access-bgm9l" (OuterVolumeSpecName: "kube-api-access-bgm9l") pod "be8293d4-b9ef-4d0f-9686-ae65c483f0f4" (UID: "be8293d4-b9ef-4d0f-9686-ae65c483f0f4"). InnerVolumeSpecName "kube-api-access-bgm9l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:51:07.129370 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:07.129286 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d11b889b-57c3-4ae5-8cce-f337308aaa6d-plugins-config-volume\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:51:07.129370 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:07.129315 2560 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/be8293d4-b9ef-4d0f-9686-ae65c483f0f4-plugins-config-volume\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:51:07.129370 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:07.129325 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tplhn\" (UniqueName: \"kubernetes.io/projected/d11b889b-57c3-4ae5-8cce-f337308aaa6d-kube-api-access-tplhn\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:51:07.129370 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:07.129333 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgm9l\" (UniqueName: \"kubernetes.io/projected/be8293d4-b9ef-4d0f-9686-ae65c483f0f4-kube-api-access-bgm9l\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:51:07.177792 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:07.177757 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:51:07.442770 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:07.442738 2560 generic.go:356] "Generic (PLEG): container finished" podID="d11b889b-57c3-4ae5-8cce-f337308aaa6d" containerID="77e1000b202d967f1a70c8578c8958fadb70c6d0b3a4b607df96d248759ba335" exitCode=0 Sep 10 13:51:07.442953 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:07.442787 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-89be92-98b7b59cf-9krl9" Sep 10 13:51:07.442953 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:07.442825 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-89be92-98b7b59cf-9krl9" event={"ID":"d11b889b-57c3-4ae5-8cce-f337308aaa6d","Type":"ContainerDied","Data":"77e1000b202d967f1a70c8578c8958fadb70c6d0b3a4b607df96d248759ba335"} Sep 10 13:51:07.442953 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:07.442881 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-89be92-98b7b59cf-9krl9" event={"ID":"d11b889b-57c3-4ae5-8cce-f337308aaa6d","Type":"ContainerDied","Data":"1d41a967e7344dc8fbc1c448832ac0d528d2ada626e9194a705eb2e4f71d9ebc"} Sep 10 13:51:07.442953 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:07.442907 2560 scope.go:117] "RemoveContainer" containerID="77e1000b202d967f1a70c8578c8958fadb70c6d0b3a4b607df96d248759ba335" Sep 10 13:51:07.444418 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:07.444371 2560 generic.go:356] "Generic (PLEG): container finished" podID="be8293d4-b9ef-4d0f-9686-ae65c483f0f4" containerID="c11c936fe3ecad3e51d1fbae5b7f87fac64267917d897e7e5875b1f55849af86" exitCode=0 Sep 10 13:51:07.444531 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:07.444434 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-89be92-6f87d675d6-jldxc" Sep 10 13:51:07.444531 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:07.444439 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-89be92-6f87d675d6-jldxc" event={"ID":"be8293d4-b9ef-4d0f-9686-ae65c483f0f4","Type":"ContainerDied","Data":"c11c936fe3ecad3e51d1fbae5b7f87fac64267917d897e7e5875b1f55849af86"} Sep 10 13:51:07.444531 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:07.444499 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-89be92-6f87d675d6-jldxc" event={"ID":"be8293d4-b9ef-4d0f-9686-ae65c483f0f4","Type":"ContainerDied","Data":"d9f8d5a77e0afd97e5c1c53348c4fe996c468d2119fcec190e1d60888b9e319b"} Sep 10 13:51:07.452636 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:07.452613 2560 scope.go:117] "RemoveContainer" containerID="77e1000b202d967f1a70c8578c8958fadb70c6d0b3a4b607df96d248759ba335" Sep 10 13:51:07.452929 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:51:07.452908 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77e1000b202d967f1a70c8578c8958fadb70c6d0b3a4b607df96d248759ba335\": container with ID starting with 77e1000b202d967f1a70c8578c8958fadb70c6d0b3a4b607df96d248759ba335 not found: ID does not exist" containerID="77e1000b202d967f1a70c8578c8958fadb70c6d0b3a4b607df96d248759ba335" Sep 10 13:51:07.453012 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:07.452936 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77e1000b202d967f1a70c8578c8958fadb70c6d0b3a4b607df96d248759ba335"} err="failed to get container status \"77e1000b202d967f1a70c8578c8958fadb70c6d0b3a4b607df96d248759ba335\": rpc error: code = NotFound desc = could not find container \"77e1000b202d967f1a70c8578c8958fadb70c6d0b3a4b607df96d248759ba335\": container with ID starting with 77e1000b202d967f1a70c8578c8958fadb70c6d0b3a4b607df96d248759ba335 not found: ID does not exist" Sep 10 13:51:07.453012 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:07.452953 2560 scope.go:117] "RemoveContainer" containerID="c11c936fe3ecad3e51d1fbae5b7f87fac64267917d897e7e5875b1f55849af86" Sep 10 13:51:07.461352 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:07.461336 2560 scope.go:117] "RemoveContainer" containerID="c11c936fe3ecad3e51d1fbae5b7f87fac64267917d897e7e5875b1f55849af86" Sep 10 13:51:07.461623 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:51:07.461605 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c11c936fe3ecad3e51d1fbae5b7f87fac64267917d897e7e5875b1f55849af86\": container with ID starting with c11c936fe3ecad3e51d1fbae5b7f87fac64267917d897e7e5875b1f55849af86 not found: ID does not exist" containerID="c11c936fe3ecad3e51d1fbae5b7f87fac64267917d897e7e5875b1f55849af86" Sep 10 13:51:07.461682 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:07.461629 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c11c936fe3ecad3e51d1fbae5b7f87fac64267917d897e7e5875b1f55849af86"} err="failed to get container status \"c11c936fe3ecad3e51d1fbae5b7f87fac64267917d897e7e5875b1f55849af86\": rpc error: code = NotFound desc = could not find container \"c11c936fe3ecad3e51d1fbae5b7f87fac64267917d897e7e5875b1f55849af86\": container with ID starting with c11c936fe3ecad3e51d1fbae5b7f87fac64267917d897e7e5875b1f55849af86 not found: ID does not exist" Sep 10 13:51:07.465868 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:07.465838 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-89be92-98b7b59cf-9krl9"] Sep 10 13:51:07.470453 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:07.470432 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-89be92-98b7b59cf-9krl9"] Sep 10 13:51:07.481735 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:07.481711 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-89be92-6f87d675d6-jldxc"] Sep 10 13:51:07.483971 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:07.483949 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-89be92-6f87d675d6-jldxc"] Sep 10 13:51:08.563968 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:08.563870 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be8293d4-b9ef-4d0f-9686-ae65c483f0f4" path="/var/lib/kubelet/pods/be8293d4-b9ef-4d0f-9686-ae65c483f0f4/volumes" Sep 10 13:51:08.564572 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:08.564438 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d11b889b-57c3-4ae5-8cce-f337308aaa6d" path="/var/lib/kubelet/pods/d11b889b-57c3-4ae5-8cce-f337308aaa6d/volumes" Sep 10 13:51:10.887603 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:10.887561 2560 scope.go:117] "RemoveContainer" containerID="b72d20a4e40e5d1dc4964fdb3000f3d4dda6b13df0d2c7226ba09de4df400d6a" Sep 10 13:51:10.896527 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:10.896504 2560 scope.go:117] "RemoveContainer" containerID="5824ebf61a42f9914c12a825b375d5bdeabff5e3adca2619c4defd91347e671b" Sep 10 13:51:10.905040 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:10.905020 2560 scope.go:117] "RemoveContainer" containerID="46f1ee33089edd877f2e1bddaf5227123f666cfbced0581d193e3bade84afc09" Sep 10 13:51:13.576329 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:13.576288 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:51:15.428067 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:15.428024 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:51:33.935808 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:33.935769 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:51:34.033079 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:34.033042 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:51:34.805982 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:34.805945 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:51:49.630836 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:49.630802 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:51:49.699163 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:49.699132 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:51:52.827537 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:51:52.827502 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:52:40.289145 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:52:40.289107 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:52:40.393439 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:52:40.388438 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:52:41.990956 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:52:41.990923 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:52:51.950908 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:52:51.950868 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:53:10.681589 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:53:10.681558 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gtqlh_52fceedd-00f2-4943-ae47-dec51216d8e4/console-operator/2.log" Sep 10 13:53:10.689205 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:53:10.689174 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gtqlh_52fceedd-00f2-4943-ae47-dec51216d8e4/console-operator/2.log" Sep 10 13:54:09.060724 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:54:09.060687 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:55:44.932193 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:55:44.932153 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:55:48.205060 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:55:48.205028 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:56:03.507100 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:03.507062 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:56:05.486156 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:05.486120 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:56:24.380526 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.380492 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6b9dddc78b-ww5k4"] Sep 10 13:56:24.380997 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.380861 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37174f02-8469-445e-be70-3bed424acbe2" containerName="istio-proxy" Sep 10 13:56:24.380997 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.380876 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="37174f02-8469-445e-be70-3bed424acbe2" containerName="istio-proxy" Sep 10 13:56:24.380997 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.380892 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38e4531c-cd84-4583-8cc8-e86e966c398e" containerName="istio-proxy" Sep 10 13:56:24.380997 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.380900 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="38e4531c-cd84-4583-8cc8-e86e966c398e" containerName="istio-proxy" Sep 10 13:56:24.380997 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.380912 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be8293d4-b9ef-4d0f-9686-ae65c483f0f4" containerName="payload-pre-processing" Sep 10 13:56:24.380997 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.380920 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="be8293d4-b9ef-4d0f-9686-ae65c483f0f4" containerName="payload-pre-processing" Sep 10 13:56:24.380997 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.380929 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37174f02-8469-445e-be70-3bed424acbe2" containerName="istio-proxy" Sep 10 13:56:24.380997 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.380936 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="37174f02-8469-445e-be70-3bed424acbe2" containerName="istio-proxy" Sep 10 13:56:24.380997 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.380951 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d11b889b-57c3-4ae5-8cce-f337308aaa6d" containerName="payload-processing" Sep 10 13:56:24.380997 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.380959 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d11b889b-57c3-4ae5-8cce-f337308aaa6d" containerName="payload-processing" Sep 10 13:56:24.380997 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.380973 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="796ef08b-1b48-4811-8a48-1aa630675f0e" containerName="payload-processing" Sep 10 13:56:24.380997 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.380978 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="796ef08b-1b48-4811-8a48-1aa630675f0e" containerName="payload-processing" Sep 10 13:56:24.380997 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.380986 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8374fc76-574f-48bd-aaf8-021f1e4e5b43" containerName="payload-pre-processing" Sep 10 13:56:24.380997 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.380991 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8374fc76-574f-48bd-aaf8-021f1e4e5b43" containerName="payload-pre-processing" Sep 10 13:56:24.380997 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.381004 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb9cd903-e9a3-4af1-b443-da43b25180ab" containerName="payload-processing" Sep 10 13:56:24.381518 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.381009 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb9cd903-e9a3-4af1-b443-da43b25180ab" containerName="payload-processing" Sep 10 13:56:24.381518 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.381015 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4cb3d14-a2aa-4a71-a1d2-61f5b61d9c7d" containerName="payload-pre-processing" Sep 10 13:56:24.381518 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.381021 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4cb3d14-a2aa-4a71-a1d2-61f5b61d9c7d" containerName="payload-pre-processing" Sep 10 13:56:24.381518 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.381028 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17221ec2-5ef2-4ef3-9889-32eb821aff1d" containerName="payload-pre-processing" Sep 10 13:56:24.381518 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.381033 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="17221ec2-5ef2-4ef3-9889-32eb821aff1d" containerName="payload-pre-processing" Sep 10 13:56:24.381518 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.381090 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="37174f02-8469-445e-be70-3bed424acbe2" containerName="istio-proxy" Sep 10 13:56:24.381518 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.381099 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="37174f02-8469-445e-be70-3bed424acbe2" containerName="istio-proxy" Sep 10 13:56:24.381518 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.381105 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb9cd903-e9a3-4af1-b443-da43b25180ab" containerName="payload-processing" Sep 10 13:56:24.381518 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.381111 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="796ef08b-1b48-4811-8a48-1aa630675f0e" containerName="payload-processing" Sep 10 13:56:24.381518 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.381119 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="17221ec2-5ef2-4ef3-9889-32eb821aff1d" containerName="payload-pre-processing" Sep 10 13:56:24.381518 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.381124 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d11b889b-57c3-4ae5-8cce-f337308aaa6d" containerName="payload-processing" Sep 10 13:56:24.381518 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.381130 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8374fc76-574f-48bd-aaf8-021f1e4e5b43" containerName="payload-pre-processing" Sep 10 13:56:24.381518 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.381135 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4cb3d14-a2aa-4a71-a1d2-61f5b61d9c7d" containerName="payload-pre-processing" Sep 10 13:56:24.381518 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.381141 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="38e4531c-cd84-4583-8cc8-e86e966c398e" containerName="istio-proxy" Sep 10 13:56:24.381518 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.381146 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="be8293d4-b9ef-4d0f-9686-ae65c483f0f4" containerName="payload-pre-processing" Sep 10 13:56:24.384171 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.384155 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6b9dddc78b-ww5k4" Sep 10 13:56:24.387724 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.387695 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Sep 10 13:56:24.387868 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.387723 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7mkhf\"" Sep 10 13:56:24.387868 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.387703 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 13:56:24.387868 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.387696 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Sep 10 13:56:24.388022 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.387880 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Sep 10 13:56:24.396050 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.396021 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b9dddc78b-ww5k4"] Sep 10 13:56:24.484396 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.484343 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a38cc0c8-da93-4ae8-84a6-fbf371fd02a1-maas-api-tls\") pod \"maas-api-6b9dddc78b-ww5k4\" (UID: \"a38cc0c8-da93-4ae8-84a6-fbf371fd02a1\") " pod="odh-ai-gateway-infra/maas-api-6b9dddc78b-ww5k4" Sep 10 13:56:24.484396 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.484397 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a38cc0c8-da93-4ae8-84a6-fbf371fd02a1-openshift-service-ca\") pod \"maas-api-6b9dddc78b-ww5k4\" (UID: \"a38cc0c8-da93-4ae8-84a6-fbf371fd02a1\") " pod="odh-ai-gateway-infra/maas-api-6b9dddc78b-ww5k4" Sep 10 13:56:24.484647 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.484544 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tdzh\" (UniqueName: \"kubernetes.io/projected/a38cc0c8-da93-4ae8-84a6-fbf371fd02a1-kube-api-access-9tdzh\") pod \"maas-api-6b9dddc78b-ww5k4\" (UID: \"a38cc0c8-da93-4ae8-84a6-fbf371fd02a1\") " pod="odh-ai-gateway-infra/maas-api-6b9dddc78b-ww5k4" Sep 10 13:56:24.484647 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.484600 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a38cc0c8-da93-4ae8-84a6-fbf371fd02a1-metrics-certs\") pod \"maas-api-6b9dddc78b-ww5k4\" (UID: \"a38cc0c8-da93-4ae8-84a6-fbf371fd02a1\") " pod="odh-ai-gateway-infra/maas-api-6b9dddc78b-ww5k4" Sep 10 13:56:24.585652 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.585618 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tdzh\" (UniqueName: \"kubernetes.io/projected/a38cc0c8-da93-4ae8-84a6-fbf371fd02a1-kube-api-access-9tdzh\") pod \"maas-api-6b9dddc78b-ww5k4\" (UID: \"a38cc0c8-da93-4ae8-84a6-fbf371fd02a1\") " pod="odh-ai-gateway-infra/maas-api-6b9dddc78b-ww5k4" Sep 10 13:56:24.585833 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.585669 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a38cc0c8-da93-4ae8-84a6-fbf371fd02a1-metrics-certs\") pod \"maas-api-6b9dddc78b-ww5k4\" (UID: \"a38cc0c8-da93-4ae8-84a6-fbf371fd02a1\") " pod="odh-ai-gateway-infra/maas-api-6b9dddc78b-ww5k4" Sep 10 13:56:24.585833 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.585725 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a38cc0c8-da93-4ae8-84a6-fbf371fd02a1-maas-api-tls\") pod \"maas-api-6b9dddc78b-ww5k4\" (UID: \"a38cc0c8-da93-4ae8-84a6-fbf371fd02a1\") " pod="odh-ai-gateway-infra/maas-api-6b9dddc78b-ww5k4" Sep 10 13:56:24.585833 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.585745 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a38cc0c8-da93-4ae8-84a6-fbf371fd02a1-openshift-service-ca\") pod \"maas-api-6b9dddc78b-ww5k4\" (UID: \"a38cc0c8-da93-4ae8-84a6-fbf371fd02a1\") " pod="odh-ai-gateway-infra/maas-api-6b9dddc78b-ww5k4" Sep 10 13:56:24.586454 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.586390 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a38cc0c8-da93-4ae8-84a6-fbf371fd02a1-openshift-service-ca\") pod \"maas-api-6b9dddc78b-ww5k4\" (UID: \"a38cc0c8-da93-4ae8-84a6-fbf371fd02a1\") " pod="odh-ai-gateway-infra/maas-api-6b9dddc78b-ww5k4" Sep 10 13:56:24.588139 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.588111 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a38cc0c8-da93-4ae8-84a6-fbf371fd02a1-metrics-certs\") pod \"maas-api-6b9dddc78b-ww5k4\" (UID: \"a38cc0c8-da93-4ae8-84a6-fbf371fd02a1\") " pod="odh-ai-gateway-infra/maas-api-6b9dddc78b-ww5k4" Sep 10 13:56:24.588303 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.588284 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a38cc0c8-da93-4ae8-84a6-fbf371fd02a1-maas-api-tls\") pod \"maas-api-6b9dddc78b-ww5k4\" (UID: \"a38cc0c8-da93-4ae8-84a6-fbf371fd02a1\") " pod="odh-ai-gateway-infra/maas-api-6b9dddc78b-ww5k4" Sep 10 13:56:24.594462 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.594437 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tdzh\" (UniqueName: \"kubernetes.io/projected/a38cc0c8-da93-4ae8-84a6-fbf371fd02a1-kube-api-access-9tdzh\") pod \"maas-api-6b9dddc78b-ww5k4\" (UID: \"a38cc0c8-da93-4ae8-84a6-fbf371fd02a1\") " pod="odh-ai-gateway-infra/maas-api-6b9dddc78b-ww5k4" Sep 10 13:56:24.696037 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.696003 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6b9dddc78b-ww5k4" Sep 10 13:56:24.973347 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.973311 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b9dddc78b-ww5k4"] Sep 10 13:56:24.974808 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:56:24.974772 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda38cc0c8_da93_4ae8_84a6_fbf371fd02a1.slice/crio-02ad31dd5efbf709319e2ef613b6a48c72c61de5c5bd0367dc0276cb1f6a82bb WatchSource:0}: Error finding container 02ad31dd5efbf709319e2ef613b6a48c72c61de5c5bd0367dc0276cb1f6a82bb: Status 404 returned error can't find the container with id 02ad31dd5efbf709319e2ef613b6a48c72c61de5c5bd0367dc0276cb1f6a82bb Sep 10 13:56:24.976168 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:24.976152 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 13:56:25.196504 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:25.196462 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6b9dddc78b-ww5k4" event={"ID":"a38cc0c8-da93-4ae8-84a6-fbf371fd02a1","Type":"ContainerStarted","Data":"02ad31dd5efbf709319e2ef613b6a48c72c61de5c5bd0367dc0276cb1f6a82bb"} Sep 10 13:56:30.229233 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:30.229196 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6b9dddc78b-ww5k4" event={"ID":"a38cc0c8-da93-4ae8-84a6-fbf371fd02a1","Type":"ContainerStarted","Data":"fc9c9e13b712210e5aa30f677288c596272c95a9b5c38a44afc09f927153d46a"} Sep 10 13:56:30.229711 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:30.229271 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6b9dddc78b-ww5k4" Sep 10 13:56:30.250817 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:30.250758 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6b9dddc78b-ww5k4" podStartSLOduration=1.771460715 podStartE2EDuration="6.25074118s" totalImagesPullingTime="4.479280465s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:56:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:56:30.249438606 +0000 UTC m=+2300.144156025" watchObservedRunningTime="2026-09-10 13:56:30.25074118 +0000 UTC m=+2300.145458598" Sep 10 13:56:32.359727 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:32.359682 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b9dddc78b-ww5k4"] Sep 10 13:56:32.360131 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:32.359918 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-6b9dddc78b-ww5k4" podUID="a38cc0c8-da93-4ae8-84a6-fbf371fd02a1" containerName="maas-api" containerID="cri-o://fc9c9e13b712210e5aa30f677288c596272c95a9b5c38a44afc09f927153d46a" gracePeriod=30 Sep 10 13:56:32.413039 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:32.413002 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-dbdb6b6cd-kmx65"] Sep 10 13:56:32.413433 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:32.413420 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38e4531c-cd84-4583-8cc8-e86e966c398e" containerName="istio-proxy" Sep 10 13:56:32.413490 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:32.413434 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="38e4531c-cd84-4583-8cc8-e86e966c398e" containerName="istio-proxy" Sep 10 13:56:32.413527 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:32.413515 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="38e4531c-cd84-4583-8cc8-e86e966c398e" containerName="istio-proxy" Sep 10 13:56:32.416664 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:32.416648 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-dbdb6b6cd-kmx65" Sep 10 13:56:32.432625 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:32.432599 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-dbdb6b6cd-kmx65"] Sep 10 13:56:32.453421 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:32.453372 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0508c43d-d388-414b-a5d1-2ebbff3c2121-openshift-service-ca\") pod \"maas-api-dbdb6b6cd-kmx65\" (UID: \"0508c43d-d388-414b-a5d1-2ebbff3c2121\") " pod="odh-ai-gateway-infra/maas-api-dbdb6b6cd-kmx65" Sep 10 13:56:32.453592 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:32.453451 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0508c43d-d388-414b-a5d1-2ebbff3c2121-maas-api-tls\") pod \"maas-api-dbdb6b6cd-kmx65\" (UID: \"0508c43d-d388-414b-a5d1-2ebbff3c2121\") " pod="odh-ai-gateway-infra/maas-api-dbdb6b6cd-kmx65" Sep 10 13:56:32.453592 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:32.453497 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0508c43d-d388-414b-a5d1-2ebbff3c2121-metrics-certs\") pod \"maas-api-dbdb6b6cd-kmx65\" (UID: \"0508c43d-d388-414b-a5d1-2ebbff3c2121\") " pod="odh-ai-gateway-infra/maas-api-dbdb6b6cd-kmx65" Sep 10 13:56:32.453679 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:32.453587 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwt9n\" (UniqueName: \"kubernetes.io/projected/0508c43d-d388-414b-a5d1-2ebbff3c2121-kube-api-access-cwt9n\") pod \"maas-api-dbdb6b6cd-kmx65\" (UID: \"0508c43d-d388-414b-a5d1-2ebbff3c2121\") " pod="odh-ai-gateway-infra/maas-api-dbdb6b6cd-kmx65" Sep 10 13:56:32.554194 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:32.554154 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0508c43d-d388-414b-a5d1-2ebbff3c2121-openshift-service-ca\") pod \"maas-api-dbdb6b6cd-kmx65\" (UID: \"0508c43d-d388-414b-a5d1-2ebbff3c2121\") " pod="odh-ai-gateway-infra/maas-api-dbdb6b6cd-kmx65" Sep 10 13:56:32.554194 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:32.554200 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0508c43d-d388-414b-a5d1-2ebbff3c2121-maas-api-tls\") pod \"maas-api-dbdb6b6cd-kmx65\" (UID: \"0508c43d-d388-414b-a5d1-2ebbff3c2121\") " pod="odh-ai-gateway-infra/maas-api-dbdb6b6cd-kmx65" Sep 10 13:56:32.554452 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:32.554253 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0508c43d-d388-414b-a5d1-2ebbff3c2121-metrics-certs\") pod \"maas-api-dbdb6b6cd-kmx65\" (UID: \"0508c43d-d388-414b-a5d1-2ebbff3c2121\") " pod="odh-ai-gateway-infra/maas-api-dbdb6b6cd-kmx65" Sep 10 13:56:32.554524 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:32.554453 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwt9n\" (UniqueName: \"kubernetes.io/projected/0508c43d-d388-414b-a5d1-2ebbff3c2121-kube-api-access-cwt9n\") pod \"maas-api-dbdb6b6cd-kmx65\" (UID: \"0508c43d-d388-414b-a5d1-2ebbff3c2121\") " pod="odh-ai-gateway-infra/maas-api-dbdb6b6cd-kmx65" Sep 10 13:56:32.554921 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:32.554893 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0508c43d-d388-414b-a5d1-2ebbff3c2121-openshift-service-ca\") pod \"maas-api-dbdb6b6cd-kmx65\" (UID: \"0508c43d-d388-414b-a5d1-2ebbff3c2121\") " pod="odh-ai-gateway-infra/maas-api-dbdb6b6cd-kmx65" Sep 10 13:56:32.557040 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:32.557018 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0508c43d-d388-414b-a5d1-2ebbff3c2121-maas-api-tls\") pod \"maas-api-dbdb6b6cd-kmx65\" (UID: \"0508c43d-d388-414b-a5d1-2ebbff3c2121\") " pod="odh-ai-gateway-infra/maas-api-dbdb6b6cd-kmx65" Sep 10 13:56:32.557167 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:32.557150 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0508c43d-d388-414b-a5d1-2ebbff3c2121-metrics-certs\") pod \"maas-api-dbdb6b6cd-kmx65\" (UID: \"0508c43d-d388-414b-a5d1-2ebbff3c2121\") " pod="odh-ai-gateway-infra/maas-api-dbdb6b6cd-kmx65" Sep 10 13:56:32.564209 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:32.564177 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwt9n\" (UniqueName: \"kubernetes.io/projected/0508c43d-d388-414b-a5d1-2ebbff3c2121-kube-api-access-cwt9n\") pod \"maas-api-dbdb6b6cd-kmx65\" (UID: \"0508c43d-d388-414b-a5d1-2ebbff3c2121\") " pod="odh-ai-gateway-infra/maas-api-dbdb6b6cd-kmx65" Sep 10 13:56:32.613869 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:32.613803 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6b9dddc78b-ww5k4" Sep 10 13:56:32.654803 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:32.654775 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a38cc0c8-da93-4ae8-84a6-fbf371fd02a1-metrics-certs\") pod \"a38cc0c8-da93-4ae8-84a6-fbf371fd02a1\" (UID: \"a38cc0c8-da93-4ae8-84a6-fbf371fd02a1\") " Sep 10 13:56:32.654963 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:32.654830 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9tdzh\" (UniqueName: \"kubernetes.io/projected/a38cc0c8-da93-4ae8-84a6-fbf371fd02a1-kube-api-access-9tdzh\") pod \"a38cc0c8-da93-4ae8-84a6-fbf371fd02a1\" (UID: \"a38cc0c8-da93-4ae8-84a6-fbf371fd02a1\") " Sep 10 13:56:32.655049 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:32.654998 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a38cc0c8-da93-4ae8-84a6-fbf371fd02a1-openshift-service-ca\") pod \"a38cc0c8-da93-4ae8-84a6-fbf371fd02a1\" (UID: \"a38cc0c8-da93-4ae8-84a6-fbf371fd02a1\") " Sep 10 13:56:32.655121 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:32.655106 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a38cc0c8-da93-4ae8-84a6-fbf371fd02a1-maas-api-tls\") pod \"a38cc0c8-da93-4ae8-84a6-fbf371fd02a1\" (UID: \"a38cc0c8-da93-4ae8-84a6-fbf371fd02a1\") " Sep 10 13:56:32.655452 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:32.655421 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a38cc0c8-da93-4ae8-84a6-fbf371fd02a1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a38cc0c8-da93-4ae8-84a6-fbf371fd02a1" (UID: "a38cc0c8-da93-4ae8-84a6-fbf371fd02a1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:56:32.657255 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:32.657198 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a38cc0c8-da93-4ae8-84a6-fbf371fd02a1-kube-api-access-9tdzh" (OuterVolumeSpecName: "kube-api-access-9tdzh") pod "a38cc0c8-da93-4ae8-84a6-fbf371fd02a1" (UID: "a38cc0c8-da93-4ae8-84a6-fbf371fd02a1"). InnerVolumeSpecName "kube-api-access-9tdzh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:56:32.657360 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:32.657286 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a38cc0c8-da93-4ae8-84a6-fbf371fd02a1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a38cc0c8-da93-4ae8-84a6-fbf371fd02a1" (UID: "a38cc0c8-da93-4ae8-84a6-fbf371fd02a1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 13:56:32.657442 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:32.657356 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a38cc0c8-da93-4ae8-84a6-fbf371fd02a1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a38cc0c8-da93-4ae8-84a6-fbf371fd02a1" (UID: "a38cc0c8-da93-4ae8-84a6-fbf371fd02a1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 13:56:32.742855 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:32.742808 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-dbdb6b6cd-kmx65" Sep 10 13:56:32.756904 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:32.756880 2560 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a38cc0c8-da93-4ae8-84a6-fbf371fd02a1-metrics-certs\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:56:32.756961 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:32.756911 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9tdzh\" (UniqueName: \"kubernetes.io/projected/a38cc0c8-da93-4ae8-84a6-fbf371fd02a1-kube-api-access-9tdzh\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:56:32.756961 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:32.756928 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a38cc0c8-da93-4ae8-84a6-fbf371fd02a1-openshift-service-ca\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:56:32.756961 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:32.756942 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a38cc0c8-da93-4ae8-84a6-fbf371fd02a1-maas-api-tls\") on node \"ip-10-0-142-118.ec2.internal\" DevicePath \"\"" Sep 10 13:56:33.021088 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:33.021059 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-dbdb6b6cd-kmx65"] Sep 10 13:56:33.022454 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:56:33.022428 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0508c43d_d388_414b_a5d1_2ebbff3c2121.slice/crio-316a27439265ef13cfb9fd698b8f008c096bfe4d2ead6570b51011ed9e95524b WatchSource:0}: Error finding container 316a27439265ef13cfb9fd698b8f008c096bfe4d2ead6570b51011ed9e95524b: Status 404 returned error can't find the container with id 316a27439265ef13cfb9fd698b8f008c096bfe4d2ead6570b51011ed9e95524b Sep 10 13:56:33.246206 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:33.246168 2560 generic.go:356] "Generic (PLEG): container finished" podID="a38cc0c8-da93-4ae8-84a6-fbf371fd02a1" containerID="fc9c9e13b712210e5aa30f677288c596272c95a9b5c38a44afc09f927153d46a" exitCode=0 Sep 10 13:56:33.246361 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:33.246215 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6b9dddc78b-ww5k4" Sep 10 13:56:33.246361 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:33.246245 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6b9dddc78b-ww5k4" event={"ID":"a38cc0c8-da93-4ae8-84a6-fbf371fd02a1","Type":"ContainerDied","Data":"fc9c9e13b712210e5aa30f677288c596272c95a9b5c38a44afc09f927153d46a"} Sep 10 13:56:33.246361 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:33.246331 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6b9dddc78b-ww5k4" event={"ID":"a38cc0c8-da93-4ae8-84a6-fbf371fd02a1","Type":"ContainerDied","Data":"02ad31dd5efbf709319e2ef613b6a48c72c61de5c5bd0367dc0276cb1f6a82bb"} Sep 10 13:56:33.246361 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:33.246360 2560 scope.go:117] "RemoveContainer" containerID="fc9c9e13b712210e5aa30f677288c596272c95a9b5c38a44afc09f927153d46a" Sep 10 13:56:33.247556 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:33.247534 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-dbdb6b6cd-kmx65" event={"ID":"0508c43d-d388-414b-a5d1-2ebbff3c2121","Type":"ContainerStarted","Data":"316a27439265ef13cfb9fd698b8f008c096bfe4d2ead6570b51011ed9e95524b"} Sep 10 13:56:33.256062 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:33.256042 2560 scope.go:117] "RemoveContainer" containerID="fc9c9e13b712210e5aa30f677288c596272c95a9b5c38a44afc09f927153d46a" Sep 10 13:56:33.256355 ip-10-0-142-118 kubenswrapper[2560]: E0910 13:56:33.256337 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc9c9e13b712210e5aa30f677288c596272c95a9b5c38a44afc09f927153d46a\": container with ID starting with fc9c9e13b712210e5aa30f677288c596272c95a9b5c38a44afc09f927153d46a not found: ID does not exist" containerID="fc9c9e13b712210e5aa30f677288c596272c95a9b5c38a44afc09f927153d46a" Sep 10 13:56:33.256393 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:33.256366 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc9c9e13b712210e5aa30f677288c596272c95a9b5c38a44afc09f927153d46a"} err="failed to get container status \"fc9c9e13b712210e5aa30f677288c596272c95a9b5c38a44afc09f927153d46a\": rpc error: code = NotFound desc = could not find container \"fc9c9e13b712210e5aa30f677288c596272c95a9b5c38a44afc09f927153d46a\": container with ID starting with fc9c9e13b712210e5aa30f677288c596272c95a9b5c38a44afc09f927153d46a not found: ID does not exist" Sep 10 13:56:33.269853 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:33.269778 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b9dddc78b-ww5k4"] Sep 10 13:56:33.274387 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:33.274360 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b9dddc78b-ww5k4"] Sep 10 13:56:34.256392 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:34.256357 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-dbdb6b6cd-kmx65" event={"ID":"0508c43d-d388-414b-a5d1-2ebbff3c2121","Type":"ContainerStarted","Data":"c11532f48c0190b9e600f162ec63e67f6a76107ec5144f0f8a47f6812e2cf14a"} Sep 10 13:56:34.256806 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:34.256442 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-dbdb6b6cd-kmx65" Sep 10 13:56:34.276979 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:34.276926 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-dbdb6b6cd-kmx65" podStartSLOduration=1.813753945 podStartE2EDuration="2.276908482s" totalImagesPullingTime="463.154537ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:56:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:56:34.274779321 +0000 UTC m=+2304.169496849" watchObservedRunningTime="2026-09-10 13:56:34.276908482 +0000 UTC m=+2304.171625900" Sep 10 13:56:34.563758 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:34.563680 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a38cc0c8-da93-4ae8-84a6-fbf371fd02a1" path="/var/lib/kubelet/pods/a38cc0c8-da93-4ae8-84a6-fbf371fd02a1/volumes" Sep 10 13:56:40.267871 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:40.267842 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-dbdb6b6cd-kmx65" Sep 10 13:56:59.901684 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:56:59.901653 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-gllqt_40bbd992-6076-4336-8062-20ac28f260a6/server/0.log" Sep 10 13:57:00.024339 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:00.024300 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-8sz42_8a279cf1-6227-43c0-98a3-9dab824bac05/manager/0.log" Sep 10 13:57:00.264796 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:00.264767 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84d444f55-84thz_710f018c-075a-428e-aed2-d031822357a4/copy-manifests/0.log" Sep 10 13:57:00.272021 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:00.271999 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84d444f55-84thz_710f018c-075a-428e-aed2-d031822357a4/manager/0.log" Sep 10 13:57:00.840029 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:00.840002 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-dbdb6b6cd-kmx65_0508c43d-d388-414b-a5d1-2ebbff3c2121/maas-api/0.log" Sep 10 13:57:02.899814 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:02.899720 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg_acf02ae3-6a55-453b-b14d-d86d21aee134/istio-proxy/0.log" Sep 10 13:57:03.021419 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:03.021367 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-5kfd2_04fbc9f1-fede-4e96-b94d-9dc07970c8a7/discovery/0.log" Sep 10 13:57:03.503515 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:03.503478 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-f27mg_88456670-7401-4573-9af4-58409e5bbd86/payload-pre-processing/0.log" Sep 10 13:57:03.746089 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:03.746041 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5787c86768-n9xzb_b19de8c5-85f7-4f5e-9011-8f38d0b80a69/router/0.log" Sep 10 13:57:04.097122 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:04.097089 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-9xkjd_e5ee4a41-7e88-44b8-ac33-879e10aca6a0/main/0.log" Sep 10 13:57:04.450790 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:04.450761 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-ggln2_2940f367-8823-4044-b6dd-29d08ad299f0/main/0.log" Sep 10 13:57:10.815943 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:10.815894 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:57:13.804756 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:13.804727 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-v2hld_faa8bfdc-9c79-4c12-8ec9-3cd8639d0793/global-pull-secret-syncer/0.log" Sep 10 13:57:13.870377 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:13.870341 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" Sep 10 13:57:14.077515 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:14.077430 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-118.ec2.internal_b43f7a99705be5e3239e1140886f9e8c/haproxy/0.log" Sep 10 13:57:14.857638 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:14.857588 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:57:20.375266 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:20.375235 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6d221cc8-d9d7-49a6-82e4-14e719a4dc96/alertmanager/0.log" Sep 10 13:57:20.401661 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:20.401635 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6d221cc8-d9d7-49a6-82e4-14e719a4dc96/config-reloader/0.log" Sep 10 13:57:20.428572 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:20.428538 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6d221cc8-d9d7-49a6-82e4-14e719a4dc96/kube-rbac-proxy-web/0.log" Sep 10 13:57:20.455501 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:20.455472 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6d221cc8-d9d7-49a6-82e4-14e719a4dc96/kube-rbac-proxy/0.log" Sep 10 13:57:20.479595 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:20.479563 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6d221cc8-d9d7-49a6-82e4-14e719a4dc96/kube-rbac-proxy-metric/0.log" Sep 10 13:57:20.511250 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:20.511215 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6d221cc8-d9d7-49a6-82e4-14e719a4dc96/prom-label-proxy/0.log" Sep 10 13:57:20.536231 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:20.536207 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6d221cc8-d9d7-49a6-82e4-14e719a4dc96/init-config-reloader/0.log" Sep 10 13:57:20.594607 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:20.594578 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-8k875_4a051c0c-418c-439b-94e1-181c0a84e86b/cluster-monitoring-operator/0.log" Sep 10 13:57:21.146993 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:21.146912 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zscnt_e6173c35-bb5d-438a-be83-6da1061b6d32/node-exporter/0.log" Sep 10 13:57:21.173721 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:21.173678 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zscnt_e6173c35-bb5d-438a-be83-6da1061b6d32/kube-rbac-proxy/0.log" Sep 10 13:57:21.218507 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:21.218478 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zscnt_e6173c35-bb5d-438a-be83-6da1061b6d32/init-textfile/0.log" Sep 10 13:57:23.159374 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:23.159346 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-qw7g8_e7fc744a-ed33-44a7-94f1-19bbd01b9c50/networking-console-plugin/0.log" Sep 10 13:57:23.802749 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:23.802716 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gtqlh_52fceedd-00f2-4943-ae47-dec51216d8e4/console-operator/2.log" Sep 10 13:57:23.808128 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:23.808101 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-gtqlh_52fceedd-00f2-4943-ae47-dec51216d8e4/console-operator/3.log" Sep 10 13:57:23.808690 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:23.808651 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-n2zxf/perf-node-gather-daemonset-8l52m"] Sep 10 13:57:23.809082 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:23.809065 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a38cc0c8-da93-4ae8-84a6-fbf371fd02a1" containerName="maas-api" Sep 10 13:57:23.809082 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:23.809083 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a38cc0c8-da93-4ae8-84a6-fbf371fd02a1" containerName="maas-api" Sep 10 13:57:23.809209 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:23.809154 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a38cc0c8-da93-4ae8-84a6-fbf371fd02a1" containerName="maas-api" Sep 10 13:57:23.812354 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:23.812337 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8l52m" Sep 10 13:57:23.814741 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:23.814722 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-n2zxf\"/\"kube-root-ca.crt\"" Sep 10 13:57:23.814852 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:23.814783 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-n2zxf\"/\"openshift-service-ca.crt\"" Sep 10 13:57:23.814852 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:23.814783 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-n2zxf\"/\"default-dockercfg-frlhp\"" Sep 10 13:57:23.822628 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:23.822605 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-n2zxf/perf-node-gather-daemonset-8l52m"] Sep 10 13:57:23.825795 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:23.825773 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b7d8cde3-f543-4e14-85d0-8f27cfd8c0f2-lib-modules\") pod \"perf-node-gather-daemonset-8l52m\" (UID: \"b7d8cde3-f543-4e14-85d0-8f27cfd8c0f2\") " pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8l52m" Sep 10 13:57:23.825912 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:23.825805 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b7d8cde3-f543-4e14-85d0-8f27cfd8c0f2-sys\") pod \"perf-node-gather-daemonset-8l52m\" (UID: \"b7d8cde3-f543-4e14-85d0-8f27cfd8c0f2\") " pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8l52m" Sep 10 13:57:23.825912 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:23.825831 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b7d8cde3-f543-4e14-85d0-8f27cfd8c0f2-proc\") pod \"perf-node-gather-daemonset-8l52m\" (UID: \"b7d8cde3-f543-4e14-85d0-8f27cfd8c0f2\") " pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8l52m" Sep 10 13:57:23.826002 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:23.825964 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2tmc\" (UniqueName: \"kubernetes.io/projected/b7d8cde3-f543-4e14-85d0-8f27cfd8c0f2-kube-api-access-x2tmc\") pod \"perf-node-gather-daemonset-8l52m\" (UID: \"b7d8cde3-f543-4e14-85d0-8f27cfd8c0f2\") " pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8l52m" Sep 10 13:57:23.826002 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:23.825993 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b7d8cde3-f543-4e14-85d0-8f27cfd8c0f2-podres\") pod \"perf-node-gather-daemonset-8l52m\" (UID: \"b7d8cde3-f543-4e14-85d0-8f27cfd8c0f2\") " pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8l52m" Sep 10 13:57:23.926681 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:23.926639 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b7d8cde3-f543-4e14-85d0-8f27cfd8c0f2-podres\") pod \"perf-node-gather-daemonset-8l52m\" (UID: \"b7d8cde3-f543-4e14-85d0-8f27cfd8c0f2\") " pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8l52m" Sep 10 13:57:23.926878 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:23.926707 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b7d8cde3-f543-4e14-85d0-8f27cfd8c0f2-lib-modules\") pod \"perf-node-gather-daemonset-8l52m\" (UID: \"b7d8cde3-f543-4e14-85d0-8f27cfd8c0f2\") " pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8l52m" Sep 10 13:57:23.926878 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:23.926728 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b7d8cde3-f543-4e14-85d0-8f27cfd8c0f2-sys\") pod \"perf-node-gather-daemonset-8l52m\" (UID: \"b7d8cde3-f543-4e14-85d0-8f27cfd8c0f2\") " pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8l52m" Sep 10 13:57:23.926878 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:23.926753 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b7d8cde3-f543-4e14-85d0-8f27cfd8c0f2-proc\") pod \"perf-node-gather-daemonset-8l52m\" (UID: \"b7d8cde3-f543-4e14-85d0-8f27cfd8c0f2\") " pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8l52m" Sep 10 13:57:23.926878 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:23.926805 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2tmc\" (UniqueName: \"kubernetes.io/projected/b7d8cde3-f543-4e14-85d0-8f27cfd8c0f2-kube-api-access-x2tmc\") pod \"perf-node-gather-daemonset-8l52m\" (UID: \"b7d8cde3-f543-4e14-85d0-8f27cfd8c0f2\") " pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8l52m" Sep 10 13:57:23.926878 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:23.926818 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b7d8cde3-f543-4e14-85d0-8f27cfd8c0f2-podres\") pod \"perf-node-gather-daemonset-8l52m\" (UID: \"b7d8cde3-f543-4e14-85d0-8f27cfd8c0f2\") " pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8l52m" Sep 10 13:57:23.926878 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:23.926847 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b7d8cde3-f543-4e14-85d0-8f27cfd8c0f2-sys\") pod \"perf-node-gather-daemonset-8l52m\" (UID: \"b7d8cde3-f543-4e14-85d0-8f27cfd8c0f2\") " pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8l52m" Sep 10 13:57:23.926878 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:23.926855 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b7d8cde3-f543-4e14-85d0-8f27cfd8c0f2-lib-modules\") pod \"perf-node-gather-daemonset-8l52m\" (UID: \"b7d8cde3-f543-4e14-85d0-8f27cfd8c0f2\") " pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8l52m" Sep 10 13:57:23.926878 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:23.926879 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b7d8cde3-f543-4e14-85d0-8f27cfd8c0f2-proc\") pod \"perf-node-gather-daemonset-8l52m\" (UID: \"b7d8cde3-f543-4e14-85d0-8f27cfd8c0f2\") " pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8l52m" Sep 10 13:57:23.937104 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:23.937076 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2tmc\" (UniqueName: \"kubernetes.io/projected/b7d8cde3-f543-4e14-85d0-8f27cfd8c0f2-kube-api-access-x2tmc\") pod \"perf-node-gather-daemonset-8l52m\" (UID: \"b7d8cde3-f543-4e14-85d0-8f27cfd8c0f2\") " pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8l52m" Sep 10 13:57:24.123298 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:24.123205 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8l52m" Sep 10 13:57:24.398741 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:24.398714 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-n2zxf/perf-node-gather-daemonset-8l52m"] Sep 10 13:57:24.400554 ip-10-0-142-118 kubenswrapper[2560]: W0910 13:57:24.400519 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb7d8cde3_f543_4e14_85d0_8f27cfd8c0f2.slice/crio-6ef1ee605ce435402584d0e4bb5c5bfa6b7aff4c06819f0a3d1522abdf92115a WatchSource:0}: Error finding container 6ef1ee605ce435402584d0e4bb5c5bfa6b7aff4c06819f0a3d1522abdf92115a: Status 404 returned error can't find the container with id 6ef1ee605ce435402584d0e4bb5c5bfa6b7aff4c06819f0a3d1522abdf92115a Sep 10 13:57:24.545624 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:24.545586 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8l52m" event={"ID":"b7d8cde3-f543-4e14-85d0-8f27cfd8c0f2","Type":"ContainerStarted","Data":"439133f63ad61f29feda369b117ff755d8c8f5f37d6cfcd7e91a7f2f7ceabe64"} Sep 10 13:57:24.545794 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:24.545675 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8l52m" event={"ID":"b7d8cde3-f543-4e14-85d0-8f27cfd8c0f2","Type":"ContainerStarted","Data":"6ef1ee605ce435402584d0e4bb5c5bfa6b7aff4c06819f0a3d1522abdf92115a"} Sep 10 13:57:24.545794 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:24.545784 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8l52m" Sep 10 13:57:24.564907 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:24.564856 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8l52m" podStartSLOduration=1.564838452 podStartE2EDuration="1.564838452s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:57:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:57:24.563110662 +0000 UTC m=+2354.457828081" watchObservedRunningTime="2026-09-10 13:57:24.564838452 +0000 UTC m=+2354.459555871" Sep 10 13:57:25.181121 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:25.181091 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-482xw_ce4cfb8d-701a-414f-99ac-5f73e57c81ab/volume-data-source-validator/0.log" Sep 10 13:57:26.259747 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:26.259718 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2tqpb_7fe46372-65c2-451c-9fba-11399f8c753d/dns/0.log" Sep 10 13:57:26.287676 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:26.287646 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2tqpb_7fe46372-65c2-451c-9fba-11399f8c753d/kube-rbac-proxy/0.log" Sep 10 13:57:26.472731 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:26.472700 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jv5qh_75c33310-0b35-4c84-be13-75a0d65d95e9/dns-node-resolver/0.log" Sep 10 13:57:27.126695 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:27.126663 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ps7jp_611016d1-55f4-4537-b5cb-2bd50af1d7fb/node-ca/0.log" Sep 10 13:57:28.120718 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:28.120674 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg_acf02ae3-6a55-453b-b14d-d86d21aee134/istio-proxy/0.log" Sep 10 13:57:28.336682 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:28.336648 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-5kfd2_04fbc9f1-fede-4e96-b94d-9dc07970c8a7/discovery/0.log" Sep 10 13:57:28.477315 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:28.477282 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-f27mg_88456670-7401-4573-9af4-58409e5bbd86/payload-pre-processing/0.log" Sep 10 13:57:28.548816 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:28.548779 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5787c86768-n9xzb_b19de8c5-85f7-4f5e-9011-8f38d0b80a69/router/0.log" Sep 10 13:57:29.488370 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:29.488339 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zgs9h_227a2714-52ab-48f6-904b-9e997ed5f3a3/serve-healthcheck-canary/0.log" Sep 10 13:57:29.984736 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:29.984680 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-mnksz_4fe9733a-2777-4507-af4b-29f330350747/insights-operator/0.log" Sep 10 13:57:29.985193 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:29.985176 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-mnksz_4fe9733a-2777-4507-af4b-29f330350747/insights-operator/1.log" Sep 10 13:57:30.019705 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:30.019673 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9242g_6f1d2f01-1377-4ca2-9b34-c70d033a0920/kube-rbac-proxy/0.log" Sep 10 13:57:30.042687 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:30.042636 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9242g_6f1d2f01-1377-4ca2-9b34-c70d033a0920/exporter/0.log" Sep 10 13:57:30.068439 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:30.068395 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9242g_6f1d2f01-1377-4ca2-9b34-c70d033a0920/extractor/0.log" Sep 10 13:57:30.563875 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:30.563841 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-n2zxf/perf-node-gather-daemonset-8l52m" Sep 10 13:57:32.842715 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:32.842697 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-gllqt_40bbd992-6076-4336-8062-20ac28f260a6/server/0.log" Sep 10 13:57:32.924176 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:32.924144 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-8sz42_8a279cf1-6227-43c0-98a3-9dab824bac05/manager/0.log" Sep 10 13:57:33.011521 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:33.011490 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84d444f55-84thz_710f018c-075a-428e-aed2-d031822357a4/manager/0.log" Sep 10 13:57:33.034970 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:33.034943 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84d444f55-84thz_710f018c-075a-428e-aed2-d031822357a4/copy-manifests/0.log" Sep 10 13:57:34.194202 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:34.194170 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg_acf02ae3-6a55-453b-b14d-d86d21aee134/istio-proxy/0.log" Sep 10 13:57:34.410666 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:34.410628 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-5kfd2_04fbc9f1-fede-4e96-b94d-9dc07970c8a7/discovery/0.log" Sep 10 13:57:34.506190 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:34.506096 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6746899c78-9fm28_3180edaa-1746-4080-9a8f-a0e53fb0afc6/manager/0.log" Sep 10 13:57:34.506589 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:34.506568 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-f27mg_88456670-7401-4573-9af4-58409e5bbd86/payload-pre-processing/0.log" Sep 10 13:57:34.559917 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:34.559889 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5787c86768-n9xzb_b19de8c5-85f7-4f5e-9011-8f38d0b80a69/router/0.log" Sep 10 13:57:36.032205 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:36.032179 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-kz4tg_acf02ae3-6a55-453b-b14d-d86d21aee134/istio-proxy/0.log" Sep 10 13:57:36.251182 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:36.251146 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-5kfd2_04fbc9f1-fede-4e96-b94d-9dc07970c8a7/discovery/0.log" Sep 10 13:57:36.365319 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:36.365244 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-f27mg_88456670-7401-4573-9af4-58409e5bbd86/payload-pre-processing/0.log" Sep 10 13:57:36.419162 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:36.419131 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5787c86768-n9xzb_b19de8c5-85f7-4f5e-9011-8f38d0b80a69/router/0.log" Sep 10 13:57:39.321360 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:39.321327 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2xql8_df9a2932-82b3-4e64-9be2-9338269186b8/konnectivity-agent/0.log" limit=1024 Sep 10 13:57:39.789458 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:39.789429 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-99m7r_d441e59a-93cc-4fd4-a3ad-df0831786472/migrator/0.log" Sep 10 13:57:39.812557 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:39.812523 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-99m7r_d441e59a-93cc-4fd4-a3ad-df0831786472/graceful-termination/0.log" Sep 10 13:57:40.194868 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:40.194835 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-4hmcg_9c97ef99-3ac3-4334-aba7-f9de4a6cccf6/kube-storage-version-migrator-operator/1.log" Sep 10 13:57:40.195527 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:40.195509 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-4hmcg_9c97ef99-3ac3-4334-aba7-f9de4a6cccf6/kube-storage-version-migrator-operator/0.log" Sep 10 13:57:41.254707 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:41.254678 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-28xnt_cc3dddf2-363d-4c1d-824f-aaca292a88f0/kube-multus-additional-cni-plugins/0.log" Sep 10 13:57:41.278987 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:41.278961 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-28xnt_cc3dddf2-363d-4c1d-824f-aaca292a88f0/egress-router-binary-copy/0.log" Sep 10 13:57:41.304417 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:41.304377 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-28xnt_cc3dddf2-363d-4c1d-824f-aaca292a88f0/cni-plugins/0.log" Sep 10 13:57:41.331733 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:41.331704 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-28xnt_cc3dddf2-363d-4c1d-824f-aaca292a88f0/bond-cni-plugin/0.log" Sep 10 13:57:41.356204 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:41.356180 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-28xnt_cc3dddf2-363d-4c1d-824f-aaca292a88f0/routeoverride-cni/0.log" Sep 10 13:57:41.382866 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:41.382836 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-28xnt_cc3dddf2-363d-4c1d-824f-aaca292a88f0/whereabouts-cni-bincopy/0.log" Sep 10 13:57:41.422385 ip-10-0-142-118 kubenswrapper[2560]: I0910 13:57:41.422355 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-28xnt_cc3dddf2-363d-4c1d-824f-aaca292a88f0/whereabouts-cni/0.log"