Sep 08 18:23:37.677961 ip-10-0-139-112 systemd[1]: Starting Kubernetes Kubelet... Sep 08 18:23:38.118871 ip-10-0-139-112 kubenswrapper[2555]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 08 18:23:38.118871 ip-10-0-139-112 kubenswrapper[2555]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 08 18:23:38.118871 ip-10-0-139-112 kubenswrapper[2555]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 08 18:23:38.118871 ip-10-0-139-112 kubenswrapper[2555]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 08 18:23:38.118871 ip-10-0-139-112 kubenswrapper[2555]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 08 18:23:38.119970 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.119883 2555 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 08 18:23:38.124899 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.124884 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 18:23:38.124899 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.124899 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 18:23:38.124965 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.124903 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 18:23:38.124965 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.124906 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 18:23:38.124965 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.124908 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 18:23:38.124965 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.124911 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 18:23:38.124965 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.124914 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 18:23:38.124965 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.124917 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 18:23:38.124965 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.124920 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 18:23:38.124965 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.124928 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 18:23:38.124965 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.124931 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 18:23:38.124965 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.124933 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 18:23:38.124965 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.124936 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 18:23:38.124965 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.124939 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 18:23:38.124965 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.124941 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 18:23:38.124965 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.124944 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 18:23:38.124965 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.124946 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 18:23:38.124965 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.124949 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 18:23:38.124965 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.124952 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 18:23:38.124965 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.124954 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 18:23:38.124965 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.124957 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 18:23:38.124965 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.124959 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 18:23:38.125436 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.124963 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 18:23:38.125436 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.124965 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 18:23:38.125436 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.124968 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 18:23:38.125436 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.124971 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 18:23:38.125436 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.124974 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 18:23:38.125436 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.124976 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 18:23:38.125436 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.124979 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 18:23:38.125436 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.124981 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 18:23:38.125436 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.124984 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 18:23:38.125436 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.124986 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 18:23:38.125436 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.124989 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 18:23:38.125436 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.124991 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 18:23:38.125436 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.124994 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 18:23:38.125436 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.124997 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 18:23:38.125436 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.124999 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 18:23:38.125436 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125002 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 18:23:38.125436 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125004 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 18:23:38.125436 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125006 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 18:23:38.125436 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125009 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 18:23:38.125436 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125012 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 18:23:38.125436 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125014 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 18:23:38.125987 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125017 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 18:23:38.125987 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125020 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 18:23:38.125987 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125022 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 18:23:38.125987 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125025 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 18:23:38.125987 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125027 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 18:23:38.125987 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125030 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 18:23:38.125987 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125032 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 18:23:38.125987 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125035 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 18:23:38.125987 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125037 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 18:23:38.125987 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125039 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 18:23:38.125987 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125042 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 18:23:38.125987 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125044 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 18:23:38.125987 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125047 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 18:23:38.125987 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125050 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 18:23:38.125987 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125053 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 18:23:38.125987 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125055 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 18:23:38.125987 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125058 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 18:23:38.125987 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125060 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 18:23:38.125987 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125062 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 18:23:38.125987 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125065 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 18:23:38.126478 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125068 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 18:23:38.126478 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125070 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 18:23:38.126478 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125073 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 18:23:38.126478 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125075 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 18:23:38.126478 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125077 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 18:23:38.126478 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125080 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 18:23:38.126478 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125082 2555 feature_gate.go:328] unrecognized feature gate: Example Sep 08 18:23:38.126478 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125085 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 18:23:38.126478 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125087 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 18:23:38.126478 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125090 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 18:23:38.126478 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125093 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 18:23:38.126478 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125096 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 18:23:38.126478 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125098 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 18:23:38.126478 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125101 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 18:23:38.126478 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125103 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 18:23:38.126478 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125105 2555 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 18:23:38.126478 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125108 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 18:23:38.126478 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125110 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 18:23:38.126478 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125113 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 18:23:38.127126 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125115 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 18:23:38.127126 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125118 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 18:23:38.127126 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125120 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 18:23:38.127126 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125123 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 18:23:38.127126 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125126 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 18:23:38.127126 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125129 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 18:23:38.127126 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125134 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 18:23:38.127126 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125138 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 18:23:38.127126 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125141 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 18:23:38.127126 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125145 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 18:23:38.127126 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125149 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 18:23:38.127126 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125151 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 18:23:38.127126 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125154 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 18:23:38.127126 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125157 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 18:23:38.127126 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125161 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 18:23:38.127126 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125164 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 18:23:38.127126 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125167 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 18:23:38.127126 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125879 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 18:23:38.127575 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125887 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 18:23:38.127575 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125890 2555 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 18:23:38.127575 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125894 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 18:23:38.127575 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125897 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 18:23:38.127575 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125901 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 18:23:38.127575 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125903 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 18:23:38.127575 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125906 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 18:23:38.127575 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125909 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 18:23:38.127575 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125912 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 18:23:38.127575 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125915 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 18:23:38.127575 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125918 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 18:23:38.127575 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125920 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 18:23:38.127575 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125923 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 18:23:38.127575 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125926 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 18:23:38.127575 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125928 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 18:23:38.127575 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125931 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 18:23:38.127575 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125933 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 18:23:38.127575 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125936 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 18:23:38.127575 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125938 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 18:23:38.127575 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125942 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 18:23:38.128106 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125944 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 18:23:38.128106 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125947 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 18:23:38.128106 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125949 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 18:23:38.128106 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125952 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 18:23:38.128106 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125955 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 18:23:38.128106 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125957 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 18:23:38.128106 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125960 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 18:23:38.128106 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125962 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 18:23:38.128106 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125965 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 18:23:38.128106 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125967 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 18:23:38.128106 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125970 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 18:23:38.128106 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125972 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 18:23:38.128106 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125975 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 18:23:38.128106 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125977 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 18:23:38.128106 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125980 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 18:23:38.128106 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125982 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 18:23:38.128106 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125985 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 18:23:38.128106 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125989 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 18:23:38.128106 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125991 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 18:23:38.128566 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125995 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 18:23:38.128566 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.125999 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 18:23:38.128566 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126002 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 18:23:38.128566 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126005 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 18:23:38.128566 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126007 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 18:23:38.128566 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126010 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 18:23:38.128566 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126012 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 18:23:38.128566 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126015 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 18:23:38.128566 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126017 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 18:23:38.128566 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126019 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 18:23:38.128566 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126023 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 18:23:38.128566 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126027 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 18:23:38.128566 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126031 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 18:23:38.128566 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126034 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 18:23:38.128566 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126037 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 18:23:38.128566 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126039 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 18:23:38.128566 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126042 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 18:23:38.128566 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126045 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 18:23:38.128566 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126047 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 18:23:38.129033 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126050 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 18:23:38.129033 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126052 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 18:23:38.129033 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126055 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 18:23:38.129033 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126058 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 18:23:38.129033 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126060 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 18:23:38.129033 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126078 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 18:23:38.129033 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126081 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 18:23:38.129033 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126084 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 18:23:38.129033 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126087 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 18:23:38.129033 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126089 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 18:23:38.129033 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126092 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 18:23:38.129033 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126094 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 18:23:38.129033 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126097 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 18:23:38.129033 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126099 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 18:23:38.129033 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126102 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 18:23:38.129033 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126105 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 18:23:38.129033 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126108 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 18:23:38.129033 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126110 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 18:23:38.129033 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126114 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 18:23:38.129033 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126117 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 18:23:38.129512 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126120 2555 feature_gate.go:328] unrecognized feature gate: Example Sep 08 18:23:38.129512 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126123 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 18:23:38.129512 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126126 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 18:23:38.129512 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126129 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 18:23:38.129512 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126132 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 18:23:38.129512 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126136 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 18:23:38.129512 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126139 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 18:23:38.129512 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126142 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 18:23:38.129512 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126145 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 18:23:38.129512 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126147 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 18:23:38.129512 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126150 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 18:23:38.129512 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126153 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 18:23:38.129512 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126156 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 18:23:38.129512 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126158 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 18:23:38.129512 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126161 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 18:23:38.129512 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126163 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 18:23:38.129512 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126167 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 18:23:38.129512 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126169 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 18:23:38.129512 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126172 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 18:23:38.129512 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.126174 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 18:23:38.130008 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127371 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 18:23:38.130008 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127377 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 18:23:38.130008 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127380 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 18:23:38.130008 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127384 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 18:23:38.130008 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127387 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 18:23:38.130008 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127389 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 18:23:38.130008 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127392 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 18:23:38.130008 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127396 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 18:23:38.130008 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127401 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 18:23:38.130008 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127404 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 18:23:38.130008 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127406 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 18:23:38.130008 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127409 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 18:23:38.130008 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127412 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 18:23:38.130008 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127414 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 18:23:38.130008 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127416 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 18:23:38.130008 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127419 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 18:23:38.130008 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127422 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 18:23:38.130008 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127425 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 18:23:38.130008 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127428 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 18:23:38.130008 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127431 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 18:23:38.130499 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127433 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 18:23:38.130499 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127436 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 18:23:38.130499 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127438 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 18:23:38.130499 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127441 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 18:23:38.130499 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127443 2555 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 18:23:38.130499 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127446 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 18:23:38.130499 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127448 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 18:23:38.130499 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127451 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 18:23:38.130499 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127454 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 18:23:38.130499 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127456 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 18:23:38.130499 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127459 2555 feature_gate.go:328] unrecognized feature gate: Example Sep 08 18:23:38.130499 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127461 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 18:23:38.130499 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127464 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 18:23:38.130499 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127467 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 18:23:38.130499 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127469 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 18:23:38.130499 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127472 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 18:23:38.130499 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127474 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 18:23:38.130499 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127477 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 18:23:38.130499 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127479 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 18:23:38.130499 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127482 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 18:23:38.130499 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127484 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 18:23:38.131024 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127487 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 18:23:38.131024 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127490 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 18:23:38.131024 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127492 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 18:23:38.131024 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127495 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 18:23:38.131024 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127498 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 18:23:38.131024 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127500 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 18:23:38.131024 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127503 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 18:23:38.131024 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127506 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 18:23:38.131024 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127509 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 18:23:38.131024 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127512 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 18:23:38.131024 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127514 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 18:23:38.131024 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127517 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 18:23:38.131024 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127519 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 18:23:38.131024 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127522 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 18:23:38.131024 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127524 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 18:23:38.131024 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127527 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 18:23:38.131024 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127529 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 18:23:38.131024 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127532 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 18:23:38.131024 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127534 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 18:23:38.131024 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127537 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 18:23:38.131546 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127540 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 18:23:38.131546 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127542 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 18:23:38.131546 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127545 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 18:23:38.131546 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127548 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 18:23:38.131546 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127550 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 18:23:38.131546 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127552 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 18:23:38.131546 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127555 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 18:23:38.131546 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127558 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 18:23:38.131546 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127560 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 18:23:38.131546 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127562 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 18:23:38.131546 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127565 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 18:23:38.131546 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127571 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 18:23:38.131546 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127574 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 18:23:38.131546 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127577 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 18:23:38.131546 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127580 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 18:23:38.131546 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127583 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 18:23:38.131546 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127586 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 18:23:38.131546 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127589 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 18:23:38.131546 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127592 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 18:23:38.132029 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127595 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 18:23:38.132029 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127598 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 18:23:38.132029 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127600 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 18:23:38.132029 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127603 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 18:23:38.132029 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127605 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 18:23:38.132029 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127608 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 18:23:38.132029 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127611 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 18:23:38.132029 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127613 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 18:23:38.132029 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127616 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 18:23:38.132029 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127618 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 18:23:38.132029 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127621 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 18:23:38.132029 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127623 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 18:23:38.132029 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127625 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 18:23:38.132029 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127628 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 18:23:38.132029 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127631 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 18:23:38.132029 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127633 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 18:23:38.132029 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127636 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 18:23:38.132029 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127638 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 18:23:38.132029 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.127642 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 18:23:38.132473 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127728 2555 flags.go:64] FLAG: --address="0.0.0.0" Sep 08 18:23:38.132473 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127759 2555 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 08 18:23:38.132473 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127766 2555 flags.go:64] FLAG: --anonymous-auth="true" Sep 08 18:23:38.132473 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127771 2555 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 08 18:23:38.132473 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127775 2555 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 08 18:23:38.132473 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127778 2555 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 08 18:23:38.132473 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127782 2555 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 08 18:23:38.132473 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127791 2555 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 08 18:23:38.132473 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127794 2555 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 08 18:23:38.132473 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127797 2555 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 08 18:23:38.132473 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127801 2555 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 08 18:23:38.132473 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127804 2555 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 08 18:23:38.132473 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127808 2555 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 08 18:23:38.132473 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127810 2555 flags.go:64] FLAG: --cgroup-root="" Sep 08 18:23:38.132473 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127813 2555 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 08 18:23:38.132473 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127816 2555 flags.go:64] FLAG: --client-ca-file="" Sep 08 18:23:38.132473 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127819 2555 flags.go:64] FLAG: --cloud-provider="external" Sep 08 18:23:38.132473 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127822 2555 flags.go:64] FLAG: --cluster-dns="[]" Sep 08 18:23:38.132473 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127826 2555 flags.go:64] FLAG: --cluster-domain="" Sep 08 18:23:38.132473 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127829 2555 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 08 18:23:38.132473 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127832 2555 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 08 18:23:38.132473 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127835 2555 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 08 18:23:38.132473 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127838 2555 flags.go:64] FLAG: --container-log-max-files="5" Sep 08 18:23:38.132473 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127842 2555 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 08 18:23:38.133112 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127844 2555 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 08 18:23:38.133112 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127847 2555 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 08 18:23:38.133112 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127851 2555 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 08 18:23:38.133112 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127854 2555 flags.go:64] FLAG: --contention-profiling="false" Sep 08 18:23:38.133112 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127858 2555 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 08 18:23:38.133112 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127861 2555 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 08 18:23:38.133112 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127864 2555 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 08 18:23:38.133112 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127867 2555 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 08 18:23:38.133112 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127871 2555 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 08 18:23:38.133112 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127874 2555 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 08 18:23:38.133112 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127877 2555 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 08 18:23:38.133112 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127880 2555 flags.go:64] FLAG: --enable-load-reader="false" Sep 08 18:23:38.133112 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127882 2555 flags.go:64] FLAG: --enable-server="true" Sep 08 18:23:38.133112 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127885 2555 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 08 18:23:38.133112 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127892 2555 flags.go:64] FLAG: --event-burst="100" Sep 08 18:23:38.133112 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127895 2555 flags.go:64] FLAG: --event-qps="50" Sep 08 18:23:38.133112 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127898 2555 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 08 18:23:38.133112 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127901 2555 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 08 18:23:38.133112 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127906 2555 flags.go:64] FLAG: --eviction-hard="" Sep 08 18:23:38.133112 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127910 2555 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 08 18:23:38.133112 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127913 2555 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 08 18:23:38.133112 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127916 2555 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 08 18:23:38.133112 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127919 2555 flags.go:64] FLAG: --eviction-soft="" Sep 08 18:23:38.133112 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127922 2555 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 08 18:23:38.133112 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127924 2555 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 08 18:23:38.133762 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127927 2555 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 08 18:23:38.133762 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127930 2555 flags.go:64] FLAG: --experimental-mounter-path="" Sep 08 18:23:38.133762 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127933 2555 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 08 18:23:38.133762 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127936 2555 flags.go:64] FLAG: --fail-swap-on="true" Sep 08 18:23:38.133762 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127939 2555 flags.go:64] FLAG: --feature-gates="" Sep 08 18:23:38.133762 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127943 2555 flags.go:64] FLAG: --file-check-frequency="20s" Sep 08 18:23:38.133762 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127946 2555 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 08 18:23:38.133762 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127949 2555 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 08 18:23:38.133762 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127952 2555 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 08 18:23:38.133762 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127955 2555 flags.go:64] FLAG: --healthz-port="10248" Sep 08 18:23:38.133762 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127958 2555 flags.go:64] FLAG: --help="false" Sep 08 18:23:38.133762 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127960 2555 flags.go:64] FLAG: --hostname-override="ip-10-0-139-112.ec2.internal" Sep 08 18:23:38.133762 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127964 2555 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 08 18:23:38.133762 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127966 2555 flags.go:64] FLAG: --http-check-frequency="20s" Sep 08 18:23:38.133762 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127969 2555 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 08 18:23:38.133762 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127972 2555 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 08 18:23:38.133762 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127976 2555 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 08 18:23:38.133762 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127978 2555 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 08 18:23:38.133762 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127981 2555 flags.go:64] FLAG: --image-service-endpoint="" Sep 08 18:23:38.133762 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127984 2555 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 08 18:23:38.133762 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127987 2555 flags.go:64] FLAG: --kube-api-burst="100" Sep 08 18:23:38.133762 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127989 2555 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 08 18:23:38.133762 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127992 2555 flags.go:64] FLAG: --kube-api-qps="50" Sep 08 18:23:38.133762 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127995 2555 flags.go:64] FLAG: --kube-reserved="" Sep 08 18:23:38.134362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.127998 2555 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 08 18:23:38.134362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128002 2555 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 08 18:23:38.134362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128006 2555 flags.go:64] FLAG: --kubelet-cgroups="" Sep 08 18:23:38.134362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128009 2555 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 08 18:23:38.134362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128011 2555 flags.go:64] FLAG: --lock-file="" Sep 08 18:23:38.134362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128014 2555 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 08 18:23:38.134362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128017 2555 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 08 18:23:38.134362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128020 2555 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 08 18:23:38.134362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128025 2555 flags.go:64] FLAG: --log-json-split-stream="false" Sep 08 18:23:38.134362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128028 2555 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 08 18:23:38.134362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128031 2555 flags.go:64] FLAG: --log-text-split-stream="false" Sep 08 18:23:38.134362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128034 2555 flags.go:64] FLAG: --logging-format="text" Sep 08 18:23:38.134362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128036 2555 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 08 18:23:38.134362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128040 2555 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 08 18:23:38.134362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128043 2555 flags.go:64] FLAG: --manifest-url="" Sep 08 18:23:38.134362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128045 2555 flags.go:64] FLAG: --manifest-url-header="" Sep 08 18:23:38.134362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128049 2555 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 08 18:23:38.134362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128052 2555 flags.go:64] FLAG: --max-open-files="1000000" Sep 08 18:23:38.134362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128056 2555 flags.go:64] FLAG: --max-pods="110" Sep 08 18:23:38.134362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128059 2555 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 08 18:23:38.134362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128062 2555 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 08 18:23:38.134362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128065 2555 flags.go:64] FLAG: --memory-manager-policy="None" Sep 08 18:23:38.134362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128068 2555 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 08 18:23:38.134362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128071 2555 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 08 18:23:38.134362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128074 2555 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 08 18:23:38.134987 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128077 2555 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 08 18:23:38.134987 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128083 2555 flags.go:64] FLAG: --node-status-max-images="50" Sep 08 18:23:38.134987 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128086 2555 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 08 18:23:38.134987 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128089 2555 flags.go:64] FLAG: --oom-score-adj="-999" Sep 08 18:23:38.134987 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128092 2555 flags.go:64] FLAG: --pod-cidr="" Sep 08 18:23:38.134987 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128095 2555 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 08 18:23:38.134987 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128100 2555 flags.go:64] FLAG: --pod-manifest-path="" Sep 08 18:23:38.134987 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128103 2555 flags.go:64] FLAG: --pod-max-pids="-1" Sep 08 18:23:38.134987 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128105 2555 flags.go:64] FLAG: --pods-per-core="0" Sep 08 18:23:38.134987 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128112 2555 flags.go:64] FLAG: --port="10250" Sep 08 18:23:38.134987 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128115 2555 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 08 18:23:38.134987 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128118 2555 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0de7fe9d833b381cd" Sep 08 18:23:38.134987 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128121 2555 flags.go:64] FLAG: --qos-reserved="" Sep 08 18:23:38.134987 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128124 2555 flags.go:64] FLAG: --read-only-port="10255" Sep 08 18:23:38.134987 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128127 2555 flags.go:64] FLAG: --register-node="true" Sep 08 18:23:38.134987 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128130 2555 flags.go:64] FLAG: --register-with-taints="" Sep 08 18:23:38.134987 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128134 2555 flags.go:64] FLAG: --registry-burst="10" Sep 08 18:23:38.134987 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128136 2555 flags.go:64] FLAG: --registry-qps="5" Sep 08 18:23:38.134987 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128139 2555 flags.go:64] FLAG: --reserved-cpus="" Sep 08 18:23:38.134987 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128142 2555 flags.go:64] FLAG: --reserved-memory="" Sep 08 18:23:38.134987 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128146 2555 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 08 18:23:38.134987 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128149 2555 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 08 18:23:38.134987 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128152 2555 flags.go:64] FLAG: --rotate-certificates="false" Sep 08 18:23:38.134987 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128154 2555 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 08 18:23:38.134987 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128157 2555 flags.go:64] FLAG: --runonce="false" Sep 08 18:23:38.135671 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128160 2555 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 08 18:23:38.135671 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128163 2555 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 08 18:23:38.135671 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128166 2555 flags.go:64] FLAG: --seccomp-default="false" Sep 08 18:23:38.135671 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128168 2555 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 08 18:23:38.135671 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128171 2555 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 08 18:23:38.135671 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128174 2555 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 08 18:23:38.135671 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128177 2555 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 08 18:23:38.135671 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128180 2555 flags.go:64] FLAG: --storage-driver-password="root" Sep 08 18:23:38.135671 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128183 2555 flags.go:64] FLAG: --storage-driver-secure="false" Sep 08 18:23:38.135671 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128186 2555 flags.go:64] FLAG: --storage-driver-table="stats" Sep 08 18:23:38.135671 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128188 2555 flags.go:64] FLAG: --storage-driver-user="root" Sep 08 18:23:38.135671 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128191 2555 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 08 18:23:38.135671 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128194 2555 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 08 18:23:38.135671 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128197 2555 flags.go:64] FLAG: --system-cgroups="" Sep 08 18:23:38.135671 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128199 2555 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 08 18:23:38.135671 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128204 2555 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 08 18:23:38.135671 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128210 2555 flags.go:64] FLAG: --tls-cert-file="" Sep 08 18:23:38.135671 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128213 2555 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 08 18:23:38.135671 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128217 2555 flags.go:64] FLAG: --tls-min-version="" Sep 08 18:23:38.135671 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128219 2555 flags.go:64] FLAG: --tls-private-key-file="" Sep 08 18:23:38.135671 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128222 2555 flags.go:64] FLAG: --topology-manager-policy="none" Sep 08 18:23:38.135671 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128225 2555 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 08 18:23:38.135671 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128228 2555 flags.go:64] FLAG: --topology-manager-scope="container" Sep 08 18:23:38.135671 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128231 2555 flags.go:64] FLAG: --v="2" Sep 08 18:23:38.135671 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128234 2555 flags.go:64] FLAG: --version="false" Sep 08 18:23:38.136298 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128238 2555 flags.go:64] FLAG: --vmodule="" Sep 08 18:23:38.136298 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128242 2555 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 08 18:23:38.136298 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.128246 2555 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 08 18:23:38.136298 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128358 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 18:23:38.136298 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128362 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 18:23:38.136298 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128365 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 18:23:38.136298 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128368 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 18:23:38.136298 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128371 2555 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 18:23:38.136298 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128374 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 18:23:38.136298 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128376 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 18:23:38.136298 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128379 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 18:23:38.136298 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128382 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 18:23:38.136298 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128385 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 18:23:38.136298 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128388 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 18:23:38.136298 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128391 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 18:23:38.136298 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128393 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 18:23:38.136298 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128396 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 18:23:38.136298 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128398 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 18:23:38.136298 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128401 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 18:23:38.136298 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128403 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 18:23:38.137120 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128406 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 18:23:38.137120 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128408 2555 feature_gate.go:328] unrecognized feature gate: Example Sep 08 18:23:38.137120 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128411 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 18:23:38.137120 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128416 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 18:23:38.137120 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128418 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 18:23:38.137120 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128421 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 18:23:38.137120 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128424 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 18:23:38.137120 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128428 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 18:23:38.137120 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128431 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 18:23:38.137120 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128433 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 18:23:38.137120 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128436 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 18:23:38.137120 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128439 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 18:23:38.137120 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128441 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 18:23:38.137120 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128444 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 18:23:38.137120 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128446 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 18:23:38.137120 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128449 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 18:23:38.137120 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128451 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 18:23:38.137120 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128454 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 18:23:38.137120 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128456 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 18:23:38.137658 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128459 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 18:23:38.137658 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128461 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 18:23:38.137658 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128464 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 18:23:38.137658 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128466 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 18:23:38.137658 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128469 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 18:23:38.137658 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128471 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 18:23:38.137658 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128474 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 18:23:38.137658 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128477 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 18:23:38.137658 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128479 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 18:23:38.137658 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128481 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 18:23:38.137658 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128484 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 18:23:38.137658 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128486 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 18:23:38.137658 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128488 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 18:23:38.137658 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128491 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 18:23:38.137658 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128493 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 18:23:38.137658 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128496 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 18:23:38.137658 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128501 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 18:23:38.137658 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128503 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 18:23:38.137658 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128506 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 18:23:38.137658 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128508 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 18:23:38.138176 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128510 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 18:23:38.138176 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128514 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 18:23:38.138176 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128517 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 18:23:38.138176 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128520 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 18:23:38.138176 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128523 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 18:23:38.138176 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128525 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 18:23:38.138176 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128528 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 18:23:38.138176 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128530 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 18:23:38.138176 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128533 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 18:23:38.138176 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128535 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 18:23:38.138176 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128537 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 18:23:38.138176 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128540 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 18:23:38.138176 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128542 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 18:23:38.138176 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128544 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 18:23:38.138176 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128547 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 18:23:38.138176 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128549 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 18:23:38.138176 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128552 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 18:23:38.138176 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128554 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 18:23:38.138176 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128556 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 18:23:38.138176 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128559 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 18:23:38.138666 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128563 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 18:23:38.138666 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128566 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 18:23:38.138666 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128569 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 18:23:38.138666 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128572 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 18:23:38.138666 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128575 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 18:23:38.138666 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128578 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 18:23:38.138666 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128581 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 18:23:38.138666 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128584 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 18:23:38.138666 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128589 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 18:23:38.138666 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128592 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 18:23:38.138666 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128594 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 18:23:38.138666 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128597 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 18:23:38.138666 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128599 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 18:23:38.138666 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128602 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 18:23:38.138666 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128604 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 18:23:38.138666 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128606 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 18:23:38.138666 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128609 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 18:23:38.138666 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128620 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 18:23:38.138666 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128622 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 18:23:38.138666 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128625 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 18:23:38.139241 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128627 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 18:23:38.139241 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128630 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 18:23:38.139241 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.128632 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 18:23:38.139241 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.129282 2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 08 18:23:38.139241 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.138222 2555 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 08 18:23:38.139241 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.138241 2555 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 08 18:23:38.139241 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.139160 2555 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 08 18:23:38.139241 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.139168 2555 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 08 18:23:38.139241 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139227 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 18:23:38.139241 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139232 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 18:23:38.139241 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139236 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 18:23:38.139241 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139239 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 18:23:38.139241 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139243 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 18:23:38.139241 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139246 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 18:23:38.139241 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139249 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 18:23:38.139620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139252 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 18:23:38.139620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139255 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 18:23:38.139620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139257 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 18:23:38.139620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139260 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 18:23:38.139620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139262 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 18:23:38.139620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139265 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 18:23:38.139620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139268 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 18:23:38.139620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139270 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 18:23:38.139620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139272 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 18:23:38.139620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139275 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 18:23:38.139620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139277 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 18:23:38.139620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139280 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 18:23:38.139620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139283 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 18:23:38.139620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139285 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 18:23:38.139620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139288 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 18:23:38.139620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139290 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 18:23:38.139620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139292 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 18:23:38.139620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139295 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 18:23:38.139620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139297 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 18:23:38.140105 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139300 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 18:23:38.140105 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139303 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 18:23:38.140105 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139305 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 18:23:38.140105 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139307 2555 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 18:23:38.140105 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139310 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 18:23:38.140105 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139312 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 18:23:38.140105 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139315 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 18:23:38.140105 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139317 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 18:23:38.140105 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139319 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 18:23:38.140105 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139331 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 18:23:38.140105 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139335 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 18:23:38.140105 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139337 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 18:23:38.140105 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139340 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 18:23:38.140105 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139342 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 18:23:38.140105 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139345 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 18:23:38.140105 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139348 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 18:23:38.140105 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139350 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 18:23:38.140105 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139353 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 18:23:38.140105 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139355 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 18:23:38.140105 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139358 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 18:23:38.140620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139360 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 18:23:38.140620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139362 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 18:23:38.140620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139365 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 18:23:38.140620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139367 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 18:23:38.140620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139370 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 18:23:38.140620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139372 2555 feature_gate.go:328] unrecognized feature gate: Example Sep 08 18:23:38.140620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139375 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 18:23:38.140620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139378 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 18:23:38.140620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139380 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 18:23:38.140620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139382 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 18:23:38.140620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139385 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 18:23:38.140620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139388 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 18:23:38.140620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139390 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 18:23:38.140620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139393 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 18:23:38.140620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139396 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 18:23:38.140620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139398 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 18:23:38.140620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139401 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 18:23:38.140620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139403 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 18:23:38.140620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139406 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 18:23:38.140620 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139408 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 18:23:38.141121 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139410 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 18:23:38.141121 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139413 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 18:23:38.141121 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139415 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 18:23:38.141121 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139418 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 18:23:38.141121 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139420 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 18:23:38.141121 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139423 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 18:23:38.141121 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139426 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 18:23:38.141121 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139429 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 18:23:38.141121 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139431 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 18:23:38.141121 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139435 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 18:23:38.141121 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139439 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 18:23:38.141121 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139442 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 18:23:38.141121 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139444 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 18:23:38.141121 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139447 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 18:23:38.141121 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139450 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 18:23:38.141121 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139453 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 18:23:38.141121 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139455 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 18:23:38.141121 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139458 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 18:23:38.141121 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139460 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 18:23:38.141585 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139463 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 18:23:38.141585 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139465 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 18:23:38.141585 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139468 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 18:23:38.141585 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139470 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 18:23:38.141585 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139473 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 18:23:38.141585 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139475 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 18:23:38.141585 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139478 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 18:23:38.141585 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139480 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 18:23:38.141585 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139483 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 18:23:38.141585 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139487 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 18:23:38.141585 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139490 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 18:23:38.141585 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139493 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 18:23:38.141585 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139497 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 18:23:38.141585 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139500 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 18:23:38.142002 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.139505 2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 08 18:23:38.142002 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139614 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 08 18:23:38.142002 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139620 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 08 18:23:38.142002 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139622 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 08 18:23:38.142002 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139625 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 08 18:23:38.142002 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139628 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 08 18:23:38.142002 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139631 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 08 18:23:38.142002 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139634 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 08 18:23:38.142002 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139637 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 08 18:23:38.142002 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139639 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 08 18:23:38.142002 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139642 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 08 18:23:38.142002 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139644 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 08 18:23:38.142002 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139647 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 08 18:23:38.142002 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139650 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 08 18:23:38.142346 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139654 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 08 18:23:38.142346 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139658 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 08 18:23:38.142346 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139661 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 08 18:23:38.142346 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139664 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 08 18:23:38.142346 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139667 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 08 18:23:38.142346 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139670 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 08 18:23:38.142346 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139673 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 08 18:23:38.142346 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139675 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 08 18:23:38.142346 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139678 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 08 18:23:38.142346 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139680 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 08 18:23:38.142346 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139699 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 08 18:23:38.142346 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139703 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 08 18:23:38.142346 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139706 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 08 18:23:38.142346 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139709 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 08 18:23:38.142346 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139711 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 08 18:23:38.142346 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139714 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 08 18:23:38.142346 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139716 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 08 18:23:38.142346 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139719 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 08 18:23:38.142346 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139721 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 08 18:23:38.142818 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139724 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 08 18:23:38.142818 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139726 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 08 18:23:38.142818 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139729 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 08 18:23:38.142818 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139732 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 08 18:23:38.142818 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139736 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 08 18:23:38.142818 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139739 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 08 18:23:38.142818 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139742 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 08 18:23:38.142818 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139745 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 08 18:23:38.142818 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139748 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 08 18:23:38.142818 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139750 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 08 18:23:38.142818 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139753 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 08 18:23:38.142818 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139755 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 08 18:23:38.142818 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139758 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 08 18:23:38.142818 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139761 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 08 18:23:38.142818 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139764 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 08 18:23:38.142818 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139766 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 08 18:23:38.142818 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139769 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 08 18:23:38.142818 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139771 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 08 18:23:38.142818 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139774 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 08 18:23:38.143262 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139776 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 08 18:23:38.143262 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139778 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 08 18:23:38.143262 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139781 2555 feature_gate.go:328] unrecognized feature gate: Example2 Sep 08 18:23:38.143262 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139784 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 08 18:23:38.143262 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139786 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 08 18:23:38.143262 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139789 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 08 18:23:38.143262 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139792 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 08 18:23:38.143262 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139794 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 08 18:23:38.143262 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139796 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 08 18:23:38.143262 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139799 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 08 18:23:38.143262 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139801 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 08 18:23:38.143262 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139804 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 08 18:23:38.143262 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139806 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 08 18:23:38.143262 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139809 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 08 18:23:38.143262 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139811 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 08 18:23:38.143262 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139814 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 08 18:23:38.143262 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139817 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 08 18:23:38.143262 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139820 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 08 18:23:38.143262 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139822 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 08 18:23:38.143262 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139825 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 08 18:23:38.143760 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139827 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 08 18:23:38.143760 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139830 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 08 18:23:38.143760 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139832 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 08 18:23:38.143760 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139834 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 08 18:23:38.143760 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139837 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 08 18:23:38.143760 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139839 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 08 18:23:38.143760 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139841 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 08 18:23:38.143760 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139844 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 08 18:23:38.143760 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139847 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 08 18:23:38.143760 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139850 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 08 18:23:38.143760 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139853 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 08 18:23:38.143760 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139856 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 08 18:23:38.143760 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139858 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 08 18:23:38.143760 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139860 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 08 18:23:38.143760 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139863 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 08 18:23:38.143760 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139865 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 08 18:23:38.143760 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139868 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 08 18:23:38.143760 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139870 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 08 18:23:38.143760 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139873 2555 feature_gate.go:328] unrecognized feature gate: Example Sep 08 18:23:38.143760 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139875 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 08 18:23:38.144243 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139878 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 08 18:23:38.144243 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139880 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 08 18:23:38.144243 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139882 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 08 18:23:38.144243 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139885 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 08 18:23:38.144243 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139887 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 08 18:23:38.144243 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139889 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 08 18:23:38.144243 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139892 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 08 18:23:38.144243 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:38.139894 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 08 18:23:38.144243 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.139899 2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 08 18:23:38.144243 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.141041 2555 server.go:962] "Client rotation is on, will bootstrap in background" Sep 08 18:23:38.145083 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.145069 2555 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 08 18:23:38.146108 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.146098 2555 server.go:1019] "Starting client certificate rotation" Sep 08 18:23:38.146219 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.146200 2555 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 08 18:23:38.146254 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.146235 2555 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 08 18:23:38.149866 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.149853 2555 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 08 18:23:38.151009 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.150984 2555 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 08 18:23:38.160121 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.160100 2555 log.go:25] "Validated CRI v1 runtime API" Sep 08 18:23:38.164224 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.164210 2555 log.go:25] "Validated CRI v1 image API" Sep 08 18:23:38.165166 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.165149 2555 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 08 18:23:38.168463 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.168438 2555 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 a775f324-7191-4167-ba16-207da056fbca:/dev/nvme0n1p3 f147c3df-ae75-4eca-9adb-ba721f8dc199:/dev/nvme0n1p4] Sep 08 18:23:38.168535 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.168462 2555 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 08 18:23:38.174965 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.174853 2555 manager.go:216] Machine: {Timestamp:2026-09-08 18:23:38.172543489 +0000 UTC m=+0.378528879 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3106226 MemoryCapacity:32812158976 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f43ed0c0e7ef2e3551a0853fe357b SystemUUID:ec2f43ed-0c0e-7ef2-e355-1a0853fe357b BootID:958d6ce6-ced4-4191-929c-1b68699d8374 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d8:76:7b:d7:21 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d8:76:7b:d7:21 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:66:5d:4d:61:71:92 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812158976 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 08 18:23:38.174965 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.174960 2555 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 08 18:23:38.175312 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.175246 2555 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 08 18:23:38.178393 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.178369 2555 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 08 18:23:38.179635 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.179445 2555 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 08 18:23:38.179800 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.179641 2555 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-112.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 08 18:23:38.181254 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.181240 2555 topology_manager.go:138] "Creating topology manager with none policy" Sep 08 18:23:38.181298 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.181257 2555 container_manager_linux.go:309] "Creating device plugin manager" Sep 08 18:23:38.181298 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.181266 2555 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 08 18:23:38.181967 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.181956 2555 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 08 18:23:38.183856 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.183845 2555 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 08 18:23:38.188405 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.188394 2555 state_mem.go:36] "Initialized new in-memory state store" Sep 08 18:23:38.188524 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.188514 2555 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 08 18:23:38.188559 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.188544 2555 kubelet.go:486] "Attempting to sync node with API server" Sep 08 18:23:38.188559 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.188556 2555 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 08 18:23:38.188611 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.188580 2555 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 08 18:23:38.188611 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.188593 2555 kubelet.go:398] "Adding apiserver pod source" Sep 08 18:23:38.188611 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.188603 2555 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 08 18:23:38.188978 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.188967 2555 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 08 18:23:38.189024 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.188986 2555 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 08 18:23:38.191460 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.191442 2555 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 08 18:23:38.193602 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.193590 2555 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 08 18:23:38.193643 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.193619 2555 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 08 18:23:38.193793 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.193782 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 08 18:23:38.193840 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.193798 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 08 18:23:38.193840 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.193804 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 08 18:23:38.193840 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.193810 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 08 18:23:38.193840 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.193817 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 08 18:23:38.193840 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.193822 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 08 18:23:38.194401 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.194391 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 08 18:23:38.194401 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.194402 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 08 18:23:38.194476 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.194408 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 08 18:23:38.194476 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.194414 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 08 18:23:38.194476 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.194422 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 08 18:23:38.194476 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.194429 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 08 18:23:38.195348 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.195339 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 08 18:23:38.195348 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.195348 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 08 18:23:38.199481 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.199466 2555 server.go:1268] "Started kubelet" Sep 08 18:23:38.199555 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.199533 2555 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 08 18:23:38.200100 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:38.200078 2555 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 08 18:23:38.200319 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.200270 2555 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-112.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 08 18:23:38.200400 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.200254 2555 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 08 18:23:38.200400 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:38.200337 2555 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-112.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 08 18:23:38.200400 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.200362 2555 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 08 18:23:38.200528 ip-10-0-139-112 systemd[1]: Started Kubernetes Kubelet. Sep 08 18:23:38.201978 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.201958 2555 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 08 18:23:38.202052 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.201986 2555 server.go:310] "Adding debug handlers to kubelet server" Sep 08 18:23:38.205938 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:38.205905 2555 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 08 18:23:38.206003 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:38.204838 2555 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-112.ec2.internal.18d36ba1756154e8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-112.ec2.internal,UID:ip-10-0-139-112.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-112.ec2.internal,},FirstTimestamp:2026-09-08 18:23:38.199438568 +0000 UTC m=+0.405423962,LastTimestamp:2026-09-08 18:23:38.199438568 +0000 UTC m=+0.405423962,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-112.ec2.internal,}" Sep 08 18:23:38.209017 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.209002 2555 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 08 18:23:38.209746 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.209731 2555 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 08 18:23:38.209996 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:38.209928 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-112.ec2.internal\" not found" Sep 08 18:23:38.210471 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.210451 2555 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 08 18:23:38.210548 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.210473 2555 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 08 18:23:38.210548 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.210483 2555 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 08 18:23:38.210647 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.210551 2555 reconstruct.go:97] "Volume reconstruction finished" Sep 08 18:23:38.210647 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.210558 2555 reconciler.go:29] "Reconciler: start to sync state" Sep 08 18:23:38.210942 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.210921 2555 factory.go:55] Registering systemd factory Sep 08 18:23:38.211030 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.210945 2555 factory.go:223] Registration of the systemd container factory successfully Sep 08 18:23:38.211368 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.211356 2555 factory.go:153] Registering CRI-O factory Sep 08 18:23:38.211408 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.211373 2555 factory.go:223] Registration of the crio container factory successfully Sep 08 18:23:38.211437 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.211418 2555 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 08 18:23:38.211437 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.211432 2555 factory.go:103] Registering Raw factory Sep 08 18:23:38.211493 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.211473 2555 manager.go:1196] Started watching for new ooms in manager Sep 08 18:23:38.211898 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.211876 2555 manager.go:349] Starting recovery of all containers Sep 08 18:23:38.212854 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:38.212830 2555 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-112.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 08 18:23:38.212941 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:38.212855 2555 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 08 18:23:38.223319 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.223148 2555 manager.go:354] Recovery completed Sep 08 18:23:38.227243 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.227229 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 18:23:38.227380 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.227352 2555 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4pbzf" Sep 08 18:23:38.229045 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.229029 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-112.ec2.internal" event="NodeHasSufficientMemory" Sep 08 18:23:38.229095 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.229063 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-112.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 18:23:38.229095 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.229079 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-112.ec2.internal" event="NodeHasSufficientPID" Sep 08 18:23:38.229586 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.229575 2555 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 08 18:23:38.229622 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.229584 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 18:23:38.229622 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.229586 2555 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 08 18:23:38.229622 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.229619 2555 state_mem.go:36] "Initialized new in-memory state store" Sep 08 18:23:38.230266 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.230253 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-112.ec2.internal" event="NodeHasSufficientMemory" Sep 08 18:23:38.230306 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.230281 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-112.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 18:23:38.230306 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.230291 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-112.ec2.internal" event="NodeHasSufficientPID" Sep 08 18:23:38.231211 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:38.231133 2555 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-112.ec2.internal.18d36ba177251ea1 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-112.ec2.internal,UID:ip-10-0-139-112.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-112.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-112.ec2.internal,},FirstTimestamp:2026-09-08 18:23:38.229046945 +0000 UTC m=+0.435032335,LastTimestamp:2026-09-08 18:23:38.229046945 +0000 UTC m=+0.435032335,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-112.ec2.internal,}" Sep 08 18:23:38.231881 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.231868 2555 policy_none.go:49] "None policy: Start" Sep 08 18:23:38.231931 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.231884 2555 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 08 18:23:38.231931 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.231894 2555 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 08 18:23:38.233866 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.233856 2555 policy_none.go:47] "Start" Sep 08 18:23:38.236740 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.236721 2555 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4pbzf" Sep 08 18:23:38.240147 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:38.240070 2555 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-112.ec2.internal.18d36ba177257252 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-112.ec2.internal,UID:ip-10-0-139-112.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-139-112.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-139-112.ec2.internal,},FirstTimestamp:2026-09-08 18:23:38.22906837 +0000 UTC m=+0.435053760,LastTimestamp:2026-09-08 18:23:38.22906837 +0000 UTC m=+0.435053760,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-112.ec2.internal,}" Sep 08 18:23:38.294165 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.280237 2555 manager.go:337] "Starting Device Plugin manager" Sep 08 18:23:38.294165 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:38.280297 2555 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 08 18:23:38.294165 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.280309 2555 server.go:87] "Starting device plugin registration server" Sep 08 18:23:38.294165 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.280586 2555 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 08 18:23:38.294165 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.280595 2555 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 08 18:23:38.294165 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.280709 2555 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 08 18:23:38.294165 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.280782 2555 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 08 18:23:38.294165 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.280792 2555 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 08 18:23:38.294165 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:38.281384 2555 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 08 18:23:38.294165 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:38.281434 2555 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-112.ec2.internal\" not found" Sep 08 18:23:38.316349 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.316281 2555 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 08 18:23:38.317511 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.317493 2555 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 08 18:23:38.317612 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.317550 2555 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 08 18:23:38.317612 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.317578 2555 kubelet.go:2448] "Starting kubelet main sync loop" Sep 08 18:23:38.317726 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:38.317668 2555 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 08 18:23:38.319862 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.319833 2555 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 18:23:38.381706 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.381660 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 18:23:38.382623 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.382607 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-112.ec2.internal" event="NodeHasSufficientMemory" Sep 08 18:23:38.382712 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.382639 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-112.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 18:23:38.382712 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.382649 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-112.ec2.internal" event="NodeHasSufficientPID" Sep 08 18:23:38.382712 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.382672 2555 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-112.ec2.internal" Sep 08 18:23:38.393729 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.393711 2555 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-112.ec2.internal" Sep 08 18:23:38.393831 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:38.393737 2555 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-112.ec2.internal\": node \"ip-10-0-139-112.ec2.internal\" not found" Sep 08 18:23:38.418043 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.418018 2555 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-112.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-112.ec2.internal"] Sep 08 18:23:38.418151 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.418096 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 18:23:38.418967 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.418948 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-112.ec2.internal" event="NodeHasSufficientMemory" Sep 08 18:23:38.419061 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.418981 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-112.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 18:23:38.419061 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.418995 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-112.ec2.internal" event="NodeHasSufficientPID" Sep 08 18:23:38.419445 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:38.419426 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-112.ec2.internal\" not found" Sep 08 18:23:38.421386 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.421370 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 18:23:38.421497 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.421479 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-112.ec2.internal" Sep 08 18:23:38.421546 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.421517 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 18:23:38.422159 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.422145 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-112.ec2.internal" event="NodeHasSufficientMemory" Sep 08 18:23:38.422225 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.422166 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-112.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 18:23:38.422225 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.422176 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-112.ec2.internal" event="NodeHasSufficientPID" Sep 08 18:23:38.422225 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.422211 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-112.ec2.internal" event="NodeHasSufficientMemory" Sep 08 18:23:38.422364 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.422233 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-112.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 18:23:38.422364 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.422248 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-112.ec2.internal" event="NodeHasSufficientPID" Sep 08 18:23:38.424480 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.424464 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-112.ec2.internal" Sep 08 18:23:38.424564 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.424493 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 18:23:38.425344 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.425327 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-112.ec2.internal" event="NodeHasSufficientMemory" Sep 08 18:23:38.425432 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.425354 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-112.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 18:23:38.425432 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.425369 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-112.ec2.internal" event="NodeHasSufficientPID" Sep 08 18:23:38.451227 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:38.451202 2555 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-112.ec2.internal\" not found" node="ip-10-0-139-112.ec2.internal" Sep 08 18:23:38.455484 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:38.455469 2555 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-112.ec2.internal\" not found" node="ip-10-0-139-112.ec2.internal" Sep 08 18:23:38.511863 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.511835 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5cd5a5d22e051259fc25dc038d5ba1fe-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-112.ec2.internal\" (UID: \"5cd5a5d22e051259fc25dc038d5ba1fe\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-112.ec2.internal" Sep 08 18:23:38.511863 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.511865 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c532783d472be20cfe3b62023aa174a5-config\") pod \"kube-apiserver-proxy-ip-10-0-139-112.ec2.internal\" (UID: \"c532783d472be20cfe3b62023aa174a5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-112.ec2.internal" Sep 08 18:23:38.512067 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.511882 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5cd5a5d22e051259fc25dc038d5ba1fe-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-112.ec2.internal\" (UID: \"5cd5a5d22e051259fc25dc038d5ba1fe\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-112.ec2.internal" Sep 08 18:23:38.519956 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:38.519940 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-112.ec2.internal\" not found" Sep 08 18:23:38.612329 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.612263 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5cd5a5d22e051259fc25dc038d5ba1fe-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-112.ec2.internal\" (UID: \"5cd5a5d22e051259fc25dc038d5ba1fe\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-112.ec2.internal" Sep 08 18:23:38.612329 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.612295 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5cd5a5d22e051259fc25dc038d5ba1fe-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-112.ec2.internal\" (UID: \"5cd5a5d22e051259fc25dc038d5ba1fe\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-112.ec2.internal" Sep 08 18:23:38.612329 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.612314 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c532783d472be20cfe3b62023aa174a5-config\") pod \"kube-apiserver-proxy-ip-10-0-139-112.ec2.internal\" (UID: \"c532783d472be20cfe3b62023aa174a5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-112.ec2.internal" Sep 08 18:23:38.612535 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.612346 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c532783d472be20cfe3b62023aa174a5-config\") pod \"kube-apiserver-proxy-ip-10-0-139-112.ec2.internal\" (UID: \"c532783d472be20cfe3b62023aa174a5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-112.ec2.internal" Sep 08 18:23:38.612535 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.612372 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5cd5a5d22e051259fc25dc038d5ba1fe-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-112.ec2.internal\" (UID: \"5cd5a5d22e051259fc25dc038d5ba1fe\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-112.ec2.internal" Sep 08 18:23:38.612535 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.612376 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5cd5a5d22e051259fc25dc038d5ba1fe-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-112.ec2.internal\" (UID: \"5cd5a5d22e051259fc25dc038d5ba1fe\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-112.ec2.internal" Sep 08 18:23:38.620398 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:38.620372 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-112.ec2.internal\" not found" Sep 08 18:23:38.721178 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:38.721146 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-112.ec2.internal\" not found" Sep 08 18:23:38.754346 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.754315 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-112.ec2.internal" Sep 08 18:23:38.758151 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:38.758128 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-112.ec2.internal" Sep 08 18:23:38.822046 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:38.822015 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-112.ec2.internal\" not found" Sep 08 18:23:38.922642 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:38.922587 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-112.ec2.internal\" not found" Sep 08 18:23:39.023117 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:39.023085 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-112.ec2.internal\" not found" Sep 08 18:23:39.123766 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:39.123725 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-112.ec2.internal\" not found" Sep 08 18:23:39.145885 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:39.145867 2555 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 08 18:23:39.146048 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:39.146032 2555 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 08 18:23:39.224416 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:39.224339 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-112.ec2.internal\" not found" Sep 08 18:23:39.239523 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:39.239493 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-07 18:18:38 +0000 UTC" deadline="2028-04-22 07:58:09.437348214 +0000 UTC" Sep 08 18:23:39.239523 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:39.239523 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14197h34m30.197829068s" Sep 08 18:23:39.312154 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:39.312125 2555 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 18:23:39.325045 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:39.325020 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-112.ec2.internal\" not found" Sep 08 18:23:39.408200 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:39.408167 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 08 18:23:39.411394 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:39.411372 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-112.ec2.internal" event="NodeHasSufficientMemory" Sep 08 18:23:39.411394 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:39.411406 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-112.ec2.internal" event="NodeHasNoDiskPressure" Sep 08 18:23:39.411563 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:39.411421 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-112.ec2.internal" event="NodeHasSufficientPID" Sep 08 18:23:39.425227 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:39.425200 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-112.ec2.internal\" not found" Sep 08 18:23:39.463613 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:39.463573 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cd5a5d22e051259fc25dc038d5ba1fe.slice/crio-a2990fbbcd281227a19b58258ccc02e4ca45c79dfebf1b04b45ed51c52aec06d WatchSource:0}: Error finding container a2990fbbcd281227a19b58258ccc02e4ca45c79dfebf1b04b45ed51c52aec06d: Status 404 returned error can't find the container with id a2990fbbcd281227a19b58258ccc02e4ca45c79dfebf1b04b45ed51c52aec06d Sep 08 18:23:39.463826 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:39.463806 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc532783d472be20cfe3b62023aa174a5.slice/crio-57b9d3313f9587039d9a2c142257657ca95518019c66c5bfd2d2e72a8869abf4 WatchSource:0}: Error finding container 57b9d3313f9587039d9a2c142257657ca95518019c66c5bfd2d2e72a8869abf4: Status 404 returned error can't find the container with id 57b9d3313f9587039d9a2c142257657ca95518019c66c5bfd2d2e72a8869abf4 Sep 08 18:23:39.469716 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:39.469699 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 08 18:23:39.481433 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:39.481378 2555 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 18:23:39.484477 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:39.484454 2555 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 18:23:39.510979 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:39.510945 2555 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-112.ec2.internal" Sep 08 18:23:39.525068 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:39.525052 2555 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 08 18:23:39.526151 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:39.526138 2555 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-112.ec2.internal" Sep 08 18:23:39.534762 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:39.534748 2555 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 08 18:23:40.190340 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.190308 2555 apiserver.go:52] "Watching apiserver" Sep 08 18:23:40.197768 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.197743 2555 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 08 18:23:40.199071 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.199051 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-qbksq","openshift-cluster-node-tuning-operator/tuned-zsfvh","openshift-dns/node-resolver-lnhcj","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-112.ec2.internal","openshift-multus/multus-8sww7","openshift-multus/multus-additional-cni-plugins-vw886","openshift-network-diagnostics/network-check-target-tqthm","openshift-network-operator/iptables-alerter-kqt8b","kube-system/konnectivity-agent-29snt","kube-system/kube-apiserver-proxy-ip-10-0-139-112.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j9b2","openshift-image-registry/node-ca-dwsrv","openshift-multus/network-metrics-daemon-qz9vk","openshift-ovn-kubernetes/ovnkube-node-cc9pd"] Sep 08 18:23:40.201781 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.201764 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qz9vk" Sep 08 18:23:40.201865 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:40.201838 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qz9vk" podUID="3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2" Sep 08 18:23:40.203988 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.203970 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.205905 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.205882 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 08 18:23:40.206009 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.205885 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 08 18:23:40.206009 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.205891 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-qsl94\"" Sep 08 18:23:40.206263 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.206231 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lnhcj" Sep 08 18:23:40.207958 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.207941 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 08 18:23:40.208152 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.208139 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 08 18:23:40.208304 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.208278 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-jx2bc\"" Sep 08 18:23:40.208451 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.208438 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbksq" Sep 08 18:23:40.208513 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:40.208493 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbksq" podUID="2744e789-a69b-4c63-b2db-2a7676c746ca" Sep 08 18:23:40.210100 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.210075 2555 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 08 18:23:40.210900 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.210773 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.212545 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.212523 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 08 18:23:40.212646 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.212524 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 08 18:23:40.212646 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.212603 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 08 18:23:40.212756 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.212741 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 08 18:23:40.212883 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.212862 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-zsjpb\"" Sep 08 18:23:40.213207 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.213192 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vw886" Sep 08 18:23:40.214913 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.214895 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 08 18:23:40.215026 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.215010 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7fqs4\"" Sep 08 18:23:40.215091 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.215077 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 08 18:23:40.215606 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.215591 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqthm" Sep 08 18:23:40.215689 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:40.215668 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqthm" podUID="deed349e-43fc-4702-aef1-b9334795fd8e" Sep 08 18:23:40.217948 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.217931 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kqt8b" Sep 08 18:23:40.219459 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.219442 2555 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 08 18:23:40.219663 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.219649 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 08 18:23:40.219859 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.219845 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 08 18:23:40.220052 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.220037 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 08 18:23:40.220095 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.220070 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-p7rvk\"" Sep 08 18:23:40.220320 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.220301 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-29snt" Sep 08 18:23:40.221888 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.221343 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dedc0ee5-f020-456a-988e-7e4833ed34a6-multus-daemon-config\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.221888 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.221388 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/20991999-23ff-42a8-b754-8c3d3e0866ab-run\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.221888 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.221554 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/20991999-23ff-42a8-b754-8c3d3e0866ab-host\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.221888 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.221632 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/20991999-23ff-42a8-b754-8c3d3e0866ab-etc-tuned\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.221888 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.221671 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/582b7b23-efed-41a0-a58f-726efcdff3a8-tmp-dir\") pod \"node-resolver-lnhcj\" (UID: \"582b7b23-efed-41a0-a58f-726efcdff3a8\") " pod="openshift-dns/node-resolver-lnhcj" Sep 08 18:23:40.221888 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.221707 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-host-run-k8s-cni-cncf-io\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.221888 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.221808 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/20991999-23ff-42a8-b754-8c3d3e0866ab-var-lib-kubelet\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.221888 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.221854 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dedc0ee5-f020-456a-988e-7e4833ed34a6-cni-binary-copy\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.221888 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.221885 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/20991999-23ff-42a8-b754-8c3d3e0866ab-lib-modules\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.222414 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.221909 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/20991999-23ff-42a8-b754-8c3d3e0866ab-etc-modprobe-d\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.222414 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.221937 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/582b7b23-efed-41a0-a58f-726efcdff3a8-hosts-file\") pod \"node-resolver-lnhcj\" (UID: \"582b7b23-efed-41a0-a58f-726efcdff3a8\") " pod="openshift-dns/node-resolver-lnhcj" Sep 08 18:23:40.222414 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.221975 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-host-var-lib-cni-multus\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.222414 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.222005 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-host-var-lib-kubelet\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.222414 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.222036 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-etc-kubernetes\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.222414 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.222063 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2-metrics-certs\") pod \"network-metrics-daemon-qz9vk\" (UID: \"3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2\") " pod="openshift-multus/network-metrics-daemon-qz9vk" Sep 08 18:23:40.222414 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.222091 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/20991999-23ff-42a8-b754-8c3d3e0866ab-etc-systemd\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.222414 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.222122 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/20991999-23ff-42a8-b754-8c3d3e0866ab-tmp\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.222414 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.222152 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-host-run-netns\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.222414 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.222182 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-host-var-lib-cni-bin\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.222414 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.222207 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-hostroot\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.222414 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.222237 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/20991999-23ff-42a8-b754-8c3d3e0866ab-etc-sysconfig\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.222414 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.222265 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20991999-23ff-42a8-b754-8c3d3e0866ab-etc-kubernetes\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.222414 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.222296 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsrpv\" (UniqueName: \"kubernetes.io/projected/582b7b23-efed-41a0-a58f-726efcdff3a8-kube-api-access-lsrpv\") pod \"node-resolver-lnhcj\" (UID: \"582b7b23-efed-41a0-a58f-726efcdff3a8\") " pod="openshift-dns/node-resolver-lnhcj" Sep 08 18:23:40.222414 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.222327 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-multus-cni-dir\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.222414 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.222353 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-multus-conf-dir\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.223198 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.222475 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgndn\" (UniqueName: \"kubernetes.io/projected/dedc0ee5-f020-456a-988e-7e4833ed34a6-kube-api-access-zgndn\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.223198 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.222538 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-host-run-multus-certs\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.223198 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.222581 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-system-cni-dir\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.223198 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.222635 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-cnibin\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.223198 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.222681 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/20991999-23ff-42a8-b754-8c3d3e0866ab-sys\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.223198 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.222740 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p6dw\" (UniqueName: \"kubernetes.io/projected/3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2-kube-api-access-8p6dw\") pod \"network-metrics-daemon-qz9vk\" (UID: \"3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2\") " pod="openshift-multus/network-metrics-daemon-qz9vk" Sep 08 18:23:40.223198 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.222773 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/20991999-23ff-42a8-b754-8c3d3e0866ab-etc-sysctl-d\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.223198 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.222804 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 08 18:23:40.223198 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.222828 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/20991999-23ff-42a8-b754-8c3d3e0866ab-etc-sysctl-conf\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.223198 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.222839 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jhtvb\"" Sep 08 18:23:40.223198 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.222864 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbs98\" (UniqueName: \"kubernetes.io/projected/20991999-23ff-42a8-b754-8c3d3e0866ab-kube-api-access-qbs98\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.223198 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.222905 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-multus-socket-dir-parent\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.223198 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.222955 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2744e789-a69b-4c63-b2db-2a7676c746ca-kubelet-config\") pod \"global-pull-secret-syncer-qbksq\" (UID: \"2744e789-a69b-4c63-b2db-2a7676c746ca\") " pod="kube-system/global-pull-secret-syncer-qbksq" Sep 08 18:23:40.223198 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.222988 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2744e789-a69b-4c63-b2db-2a7676c746ca-dbus\") pod \"global-pull-secret-syncer-qbksq\" (UID: \"2744e789-a69b-4c63-b2db-2a7676c746ca\") " pod="kube-system/global-pull-secret-syncer-qbksq" Sep 08 18:23:40.223198 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.223003 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 08 18:23:40.223198 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.223019 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2744e789-a69b-4c63-b2db-2a7676c746ca-original-pull-secret\") pod \"global-pull-secret-syncer-qbksq\" (UID: \"2744e789-a69b-4c63-b2db-2a7676c746ca\") " pod="kube-system/global-pull-secret-syncer-qbksq" Sep 08 18:23:40.227201 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.226777 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.228812 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.228792 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 08 18:23:40.229138 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.229119 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 08 18:23:40.229239 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.229144 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qbh94\"" Sep 08 18:23:40.229351 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.229331 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 08 18:23:40.229508 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.229492 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 08 18:23:40.229621 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.229539 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 08 18:23:40.229802 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.229759 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 08 18:23:40.230013 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.229988 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j9b2" Sep 08 18:23:40.232553 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.232492 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dwsrv" Sep 08 18:23:40.234760 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.234732 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 08 18:23:40.235024 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.235001 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 08 18:23:40.235113 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.235099 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 08 18:23:40.235330 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.235311 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-tqxjc\"" Sep 08 18:23:40.235399 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.235336 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 08 18:23:40.235643 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.235611 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-c6gtv\"" Sep 08 18:23:40.235858 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.235842 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 08 18:23:40.236340 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.236153 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 08 18:23:40.246488 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.246464 2555 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-vjwfh" Sep 08 18:23:40.255099 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.254740 2555 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-vjwfh" Sep 08 18:23:40.311727 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.311688 2555 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 08 18:23:40.323692 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.323662 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/20991999-23ff-42a8-b754-8c3d3e0866ab-host\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.323857 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.323704 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/20991999-23ff-42a8-b754-8c3d3e0866ab-etc-tuned\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.323857 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.323782 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/20991999-23ff-42a8-b754-8c3d3e0866ab-host\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.323857 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.323829 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/582b7b23-efed-41a0-a58f-726efcdff3a8-tmp-dir\") pod \"node-resolver-lnhcj\" (UID: \"582b7b23-efed-41a0-a58f-726efcdff3a8\") " pod="openshift-dns/node-resolver-lnhcj" Sep 08 18:23:40.324029 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.323860 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-host-run-k8s-cni-cncf-io\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.324029 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.323888 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f5b51dfb-01a9-4412-bd5e-8a84b94035d6-host-slash\") pod \"iptables-alerter-kqt8b\" (UID: \"f5b51dfb-01a9-4412-bd5e-8a84b94035d6\") " pod="openshift-network-operator/iptables-alerter-kqt8b" Sep 08 18:23:40.324029 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.323915 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9136afd-27d0-4789-925d-0f7934e8f79b-system-cni-dir\") pod \"multus-additional-cni-plugins-vw886\" (UID: \"a9136afd-27d0-4789-925d-0f7934e8f79b\") " pod="openshift-multus/multus-additional-cni-plugins-vw886" Sep 08 18:23:40.324029 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.323944 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cfadca84-d487-4e8d-8c7c-12369bab945d-agent-certs\") pod \"konnectivity-agent-29snt\" (UID: \"cfadca84-d487-4e8d-8c7c-12369bab945d\") " pod="kube-system/konnectivity-agent-29snt" Sep 08 18:23:40.324029 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.323964 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-host-run-k8s-cni-cncf-io\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.324029 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.323972 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-host-kubelet\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.324029 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324022 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/20991999-23ff-42a8-b754-8c3d3e0866ab-var-lib-kubelet\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.324436 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324052 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4e8f614b-fe0a-407d-8701-06691bcd46c7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2j9b2\" (UID: \"4e8f614b-fe0a-407d-8701-06691bcd46c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j9b2" Sep 08 18:23:40.324436 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324080 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/20991999-23ff-42a8-b754-8c3d3e0866ab-etc-modprobe-d\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.324436 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324106 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-host-var-lib-kubelet\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.324436 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324110 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/20991999-23ff-42a8-b754-8c3d3e0866ab-var-lib-kubelet\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.324436 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324131 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-host-slash\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.324436 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324156 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-env-overrides\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.324436 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324161 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/582b7b23-efed-41a0-a58f-726efcdff3a8-tmp-dir\") pod \"node-resolver-lnhcj\" (UID: \"582b7b23-efed-41a0-a58f-726efcdff3a8\") " pod="openshift-dns/node-resolver-lnhcj" Sep 08 18:23:40.324436 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324179 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/20991999-23ff-42a8-b754-8c3d3e0866ab-etc-systemd\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.324436 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324204 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-host-run-netns\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.324436 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324230 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-host-var-lib-cni-bin\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.324436 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324237 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-host-var-lib-kubelet\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.324436 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324254 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-hostroot\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.324436 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324254 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/20991999-23ff-42a8-b754-8c3d3e0866ab-etc-modprobe-d\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.324436 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324280 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a9136afd-27d0-4789-925d-0f7934e8f79b-cnibin\") pod \"multus-additional-cni-plugins-vw886\" (UID: \"a9136afd-27d0-4789-925d-0f7934e8f79b\") " pod="openshift-multus/multus-additional-cni-plugins-vw886" Sep 08 18:23:40.324436 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324288 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/20991999-23ff-42a8-b754-8c3d3e0866ab-etc-systemd\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.324436 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324308 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a9136afd-27d0-4789-925d-0f7934e8f79b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vw886\" (UID: \"a9136afd-27d0-4789-925d-0f7934e8f79b\") " pod="openshift-multus/multus-additional-cni-plugins-vw886" Sep 08 18:23:40.324436 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324336 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-host-run-netns\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.325134 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324337 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slhhp\" (UniqueName: \"kubernetes.io/projected/a9136afd-27d0-4789-925d-0f7934e8f79b-kube-api-access-slhhp\") pod \"multus-additional-cni-plugins-vw886\" (UID: \"a9136afd-27d0-4789-925d-0f7934e8f79b\") " pod="openshift-multus/multus-additional-cni-plugins-vw886" Sep 08 18:23:40.325134 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324358 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-hostroot\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.325134 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324310 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-host-var-lib-cni-bin\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.325134 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324364 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-var-lib-openvswitch\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.325134 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324395 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-systemd-units\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.325134 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324422 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgndn\" (UniqueName: \"kubernetes.io/projected/dedc0ee5-f020-456a-988e-7e4833ed34a6-kube-api-access-zgndn\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.325134 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324444 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-host-cni-netd\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.325134 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324471 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.325134 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324604 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-host-run-multus-certs\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.325134 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324642 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-ovnkube-config\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.325134 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324733 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-system-cni-dir\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.325134 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324743 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-host-run-multus-certs\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.325134 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324759 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-cnibin\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.325134 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324784 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/20991999-23ff-42a8-b754-8c3d3e0866ab-etc-sysctl-conf\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.325134 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324808 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbs98\" (UniqueName: \"kubernetes.io/projected/20991999-23ff-42a8-b754-8c3d3e0866ab-kube-api-access-qbs98\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.325134 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324834 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-host-cni-bin\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.325134 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324859 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2744e789-a69b-4c63-b2db-2a7676c746ca-original-pull-secret\") pod \"global-pull-secret-syncer-qbksq\" (UID: \"2744e789-a69b-4c63-b2db-2a7676c746ca\") " pod="kube-system/global-pull-secret-syncer-qbksq" Sep 08 18:23:40.325794 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324871 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-system-cni-dir\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.325794 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324887 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dedc0ee5-f020-456a-988e-7e4833ed34a6-multus-daemon-config\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.325794 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324930 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e5d99ca2-78e9-43bb-946d-a944ffd92d62-host\") pod \"node-ca-dwsrv\" (UID: \"e5d99ca2-78e9-43bb-946d-a944ffd92d62\") " pod="openshift-image-registry/node-ca-dwsrv" Sep 08 18:23:40.325794 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324955 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/20991999-23ff-42a8-b754-8c3d3e0866ab-run\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.325794 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324967 2555 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 08 18:23:40.325794 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324979 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a9136afd-27d0-4789-925d-0f7934e8f79b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vw886\" (UID: \"a9136afd-27d0-4789-925d-0f7934e8f79b\") " pod="openshift-multus/multus-additional-cni-plugins-vw886" Sep 08 18:23:40.325794 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.324999 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-etc-openvswitch\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.325794 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.325018 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-run-openvswitch\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.325794 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.325043 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4e8f614b-fe0a-407d-8701-06691bcd46c7-socket-dir\") pod \"aws-ebs-csi-driver-node-2j9b2\" (UID: \"4e8f614b-fe0a-407d-8701-06691bcd46c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j9b2" Sep 08 18:23:40.325794 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.325068 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5csb7\" (UniqueName: \"kubernetes.io/projected/e5d99ca2-78e9-43bb-946d-a944ffd92d62-kube-api-access-5csb7\") pod \"node-ca-dwsrv\" (UID: \"e5d99ca2-78e9-43bb-946d-a944ffd92d62\") " pod="openshift-image-registry/node-ca-dwsrv" Sep 08 18:23:40.325794 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.325086 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dedc0ee5-f020-456a-988e-7e4833ed34a6-cni-binary-copy\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.325794 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.325112 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7h8f\" (UniqueName: \"kubernetes.io/projected/f5b51dfb-01a9-4412-bd5e-8a84b94035d6-kube-api-access-v7h8f\") pod \"iptables-alerter-kqt8b\" (UID: \"f5b51dfb-01a9-4412-bd5e-8a84b94035d6\") " pod="openshift-network-operator/iptables-alerter-kqt8b" Sep 08 18:23:40.325794 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.325131 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a9136afd-27d0-4789-925d-0f7934e8f79b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vw886\" (UID: \"a9136afd-27d0-4789-925d-0f7934e8f79b\") " pod="openshift-multus/multus-additional-cni-plugins-vw886" Sep 08 18:23:40.325794 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.325382 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-cnibin\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.325794 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.325492 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/20991999-23ff-42a8-b754-8c3d3e0866ab-etc-sysctl-conf\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.325794 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:40.325512 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 18:23:40.325794 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.325596 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/20991999-23ff-42a8-b754-8c3d3e0866ab-lib-modules\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.325794 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.325613 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/20991999-23ff-42a8-b754-8c3d3e0866ab-run\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.326517 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.325626 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/582b7b23-efed-41a0-a58f-726efcdff3a8-hosts-file\") pod \"node-resolver-lnhcj\" (UID: \"582b7b23-efed-41a0-a58f-726efcdff3a8\") " pod="openshift-dns/node-resolver-lnhcj" Sep 08 18:23:40.326517 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.325650 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-host-var-lib-cni-multus\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.326517 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.325675 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-etc-kubernetes\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.326517 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.325700 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a9136afd-27d0-4789-925d-0f7934e8f79b-cni-binary-copy\") pod \"multus-additional-cni-plugins-vw886\" (UID: \"a9136afd-27d0-4789-925d-0f7934e8f79b\") " pod="openshift-multus/multus-additional-cni-plugins-vw886" Sep 08 18:23:40.326517 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.325749 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-host-run-ovn-kubernetes\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.326517 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:40.325827 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2744e789-a69b-4c63-b2db-2a7676c746ca-original-pull-secret podName:2744e789-a69b-4c63-b2db-2a7676c746ca nodeName:}" failed. No retries permitted until 2026-09-08 18:23:40.825781386 +0000 UTC m=+3.031766764 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2744e789-a69b-4c63-b2db-2a7676c746ca-original-pull-secret") pod "global-pull-secret-syncer-qbksq" (UID: "2744e789-a69b-4c63-b2db-2a7676c746ca") : object "kube-system"/"original-pull-secret" not registered Sep 08 18:23:40.326517 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.325830 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-host-var-lib-cni-multus\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.326517 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.325860 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-etc-kubernetes\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.326517 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.325901 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-ovn-node-metrics-cert\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.326517 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.325919 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/582b7b23-efed-41a0-a58f-726efcdff3a8-hosts-file\") pod \"node-resolver-lnhcj\" (UID: \"582b7b23-efed-41a0-a58f-726efcdff3a8\") " pod="openshift-dns/node-resolver-lnhcj" Sep 08 18:23:40.326517 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.325988 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2-metrics-certs\") pod \"network-metrics-daemon-qz9vk\" (UID: \"3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2\") " pod="openshift-multus/network-metrics-daemon-qz9vk" Sep 08 18:23:40.326517 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.325993 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/20991999-23ff-42a8-b754-8c3d3e0866ab-lib-modules\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.326517 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.326032 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/20991999-23ff-42a8-b754-8c3d3e0866ab-tmp\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.326517 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:40.326076 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 18:23:40.326517 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.326096 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dedc0ee5-f020-456a-988e-7e4833ed34a6-cni-binary-copy\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.326517 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.326111 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mdvx\" (UniqueName: \"kubernetes.io/projected/deed349e-43fc-4702-aef1-b9334795fd8e-kube-api-access-9mdvx\") pod \"network-check-target-tqthm\" (UID: \"deed349e-43fc-4702-aef1-b9334795fd8e\") " pod="openshift-network-diagnostics/network-check-target-tqthm" Sep 08 18:23:40.326517 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.326141 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-host-run-netns\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.327565 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.326172 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-run-ovn\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.327565 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:40.326197 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2-metrics-certs podName:3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2 nodeName:}" failed. No retries permitted until 2026-09-08 18:23:40.826181119 +0000 UTC m=+3.032166496 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2-metrics-certs") pod "network-metrics-daemon-qz9vk" (UID: "3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 18:23:40.327565 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.326229 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-ovnkube-script-lib\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.327565 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.326286 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cfadca84-d487-4e8d-8c7c-12369bab945d-konnectivity-ca\") pod \"konnectivity-agent-29snt\" (UID: \"cfadca84-d487-4e8d-8c7c-12369bab945d\") " pod="kube-system/konnectivity-agent-29snt" Sep 08 18:23:40.327565 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.326303 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/20991999-23ff-42a8-b754-8c3d3e0866ab-etc-sysconfig\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.327565 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.326327 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20991999-23ff-42a8-b754-8c3d3e0866ab-etc-kubernetes\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.327565 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.326358 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsrpv\" (UniqueName: \"kubernetes.io/projected/582b7b23-efed-41a0-a58f-726efcdff3a8-kube-api-access-lsrpv\") pod \"node-resolver-lnhcj\" (UID: \"582b7b23-efed-41a0-a58f-726efcdff3a8\") " pod="openshift-dns/node-resolver-lnhcj" Sep 08 18:23:40.327565 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.326397 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-multus-cni-dir\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.327565 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.326422 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-multus-conf-dir\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.327565 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.326447 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-node-log\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.327565 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.326473 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxps6\" (UniqueName: \"kubernetes.io/projected/4e8f614b-fe0a-407d-8701-06691bcd46c7-kube-api-access-qxps6\") pod \"aws-ebs-csi-driver-node-2j9b2\" (UID: \"4e8f614b-fe0a-407d-8701-06691bcd46c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j9b2" Sep 08 18:23:40.327565 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.326498 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-run-systemd\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.327565 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.326481 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-112.ec2.internal" event={"ID":"5cd5a5d22e051259fc25dc038d5ba1fe","Type":"ContainerStarted","Data":"a2990fbbcd281227a19b58258ccc02e4ca45c79dfebf1b04b45ed51c52aec06d"} Sep 08 18:23:40.327565 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.326521 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4e8f614b-fe0a-407d-8701-06691bcd46c7-registration-dir\") pod \"aws-ebs-csi-driver-node-2j9b2\" (UID: \"4e8f614b-fe0a-407d-8701-06691bcd46c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j9b2" Sep 08 18:23:40.327565 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.326573 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20991999-23ff-42a8-b754-8c3d3e0866ab-etc-kubernetes\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.327565 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.326601 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4e8f614b-fe0a-407d-8701-06691bcd46c7-etc-selinux\") pod \"aws-ebs-csi-driver-node-2j9b2\" (UID: \"4e8f614b-fe0a-407d-8701-06691bcd46c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j9b2" Sep 08 18:23:40.328366 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.326629 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e5d99ca2-78e9-43bb-946d-a944ffd92d62-serviceca\") pod \"node-ca-dwsrv\" (UID: \"e5d99ca2-78e9-43bb-946d-a944ffd92d62\") " pod="openshift-image-registry/node-ca-dwsrv" Sep 08 18:23:40.328366 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.326633 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-multus-conf-dir\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.328366 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.326658 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/20991999-23ff-42a8-b754-8c3d3e0866ab-sys\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.328366 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.326683 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/20991999-23ff-42a8-b754-8c3d3e0866ab-etc-sysconfig\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.328366 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.326685 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p6dw\" (UniqueName: \"kubernetes.io/projected/3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2-kube-api-access-8p6dw\") pod \"network-metrics-daemon-qz9vk\" (UID: \"3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2\") " pod="openshift-multus/network-metrics-daemon-qz9vk" Sep 08 18:23:40.328366 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.326753 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/20991999-23ff-42a8-b754-8c3d3e0866ab-etc-sysctl-d\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.328366 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.326781 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-multus-socket-dir-parent\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.328366 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.326807 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/20991999-23ff-42a8-b754-8c3d3e0866ab-sys\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.328366 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.326810 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f5b51dfb-01a9-4412-bd5e-8a84b94035d6-iptables-alerter-script\") pod \"iptables-alerter-kqt8b\" (UID: \"f5b51dfb-01a9-4412-bd5e-8a84b94035d6\") " pod="openshift-network-operator/iptables-alerter-kqt8b" Sep 08 18:23:40.328366 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.326837 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-log-socket\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.328366 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.326875 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjg9s\" (UniqueName: \"kubernetes.io/projected/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-kube-api-access-kjg9s\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.328366 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.326892 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-multus-cni-dir\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.328366 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.326900 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4e8f614b-fe0a-407d-8701-06691bcd46c7-sys-fs\") pod \"aws-ebs-csi-driver-node-2j9b2\" (UID: \"4e8f614b-fe0a-407d-8701-06691bcd46c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j9b2" Sep 08 18:23:40.328366 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.326931 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2744e789-a69b-4c63-b2db-2a7676c746ca-kubelet-config\") pod \"global-pull-secret-syncer-qbksq\" (UID: \"2744e789-a69b-4c63-b2db-2a7676c746ca\") " pod="kube-system/global-pull-secret-syncer-qbksq" Sep 08 18:23:40.328366 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.326955 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2744e789-a69b-4c63-b2db-2a7676c746ca-dbus\") pod \"global-pull-secret-syncer-qbksq\" (UID: \"2744e789-a69b-4c63-b2db-2a7676c746ca\") " pod="kube-system/global-pull-secret-syncer-qbksq" Sep 08 18:23:40.328366 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.326980 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/20991999-23ff-42a8-b754-8c3d3e0866ab-etc-sysctl-d\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.328366 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.327006 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4e8f614b-fe0a-407d-8701-06691bcd46c7-device-dir\") pod \"aws-ebs-csi-driver-node-2j9b2\" (UID: \"4e8f614b-fe0a-407d-8701-06691bcd46c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j9b2" Sep 08 18:23:40.329210 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.327042 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2744e789-a69b-4c63-b2db-2a7676c746ca-dbus\") pod \"global-pull-secret-syncer-qbksq\" (UID: \"2744e789-a69b-4c63-b2db-2a7676c746ca\") " pod="kube-system/global-pull-secret-syncer-qbksq" Sep 08 18:23:40.329210 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.327101 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dedc0ee5-f020-456a-988e-7e4833ed34a6-multus-socket-dir-parent\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.329210 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.327145 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2744e789-a69b-4c63-b2db-2a7676c746ca-kubelet-config\") pod \"global-pull-secret-syncer-qbksq\" (UID: \"2744e789-a69b-4c63-b2db-2a7676c746ca\") " pod="kube-system/global-pull-secret-syncer-qbksq" Sep 08 18:23:40.329210 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.327345 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dedc0ee5-f020-456a-988e-7e4833ed34a6-multus-daemon-config\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.329210 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.328527 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-112.ec2.internal" event={"ID":"c532783d472be20cfe3b62023aa174a5","Type":"ContainerStarted","Data":"57b9d3313f9587039d9a2c142257657ca95518019c66c5bfd2d2e72a8869abf4"} Sep 08 18:23:40.330375 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.330347 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/20991999-23ff-42a8-b754-8c3d3e0866ab-tmp\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.341685 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.341660 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgndn\" (UniqueName: \"kubernetes.io/projected/dedc0ee5-f020-456a-988e-7e4833ed34a6-kube-api-access-zgndn\") pod \"multus-8sww7\" (UID: \"dedc0ee5-f020-456a-988e-7e4833ed34a6\") " pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.341900 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.341880 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/20991999-23ff-42a8-b754-8c3d3e0866ab-etc-tuned\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.351014 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.350973 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsrpv\" (UniqueName: \"kubernetes.io/projected/582b7b23-efed-41a0-a58f-726efcdff3a8-kube-api-access-lsrpv\") pod \"node-resolver-lnhcj\" (UID: \"582b7b23-efed-41a0-a58f-726efcdff3a8\") " pod="openshift-dns/node-resolver-lnhcj" Sep 08 18:23:40.351096 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.351041 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbs98\" (UniqueName: \"kubernetes.io/projected/20991999-23ff-42a8-b754-8c3d3e0866ab-kube-api-access-qbs98\") pod \"tuned-zsfvh\" (UID: \"20991999-23ff-42a8-b754-8c3d3e0866ab\") " pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.354019 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.353996 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p6dw\" (UniqueName: \"kubernetes.io/projected/3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2-kube-api-access-8p6dw\") pod \"network-metrics-daemon-qz9vk\" (UID: \"3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2\") " pod="openshift-multus/network-metrics-daemon-qz9vk" Sep 08 18:23:40.361893 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.361870 2555 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 08 18:23:40.427596 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.427561 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e5d99ca2-78e9-43bb-946d-a944ffd92d62-host\") pod \"node-ca-dwsrv\" (UID: \"e5d99ca2-78e9-43bb-946d-a944ffd92d62\") " pod="openshift-image-registry/node-ca-dwsrv" Sep 08 18:23:40.427596 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.427594 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a9136afd-27d0-4789-925d-0f7934e8f79b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vw886\" (UID: \"a9136afd-27d0-4789-925d-0f7934e8f79b\") " pod="openshift-multus/multus-additional-cni-plugins-vw886" Sep 08 18:23:40.427849 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.427612 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-etc-openvswitch\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.427849 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.427627 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-run-openvswitch\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.427849 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.427643 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4e8f614b-fe0a-407d-8701-06691bcd46c7-socket-dir\") pod \"aws-ebs-csi-driver-node-2j9b2\" (UID: \"4e8f614b-fe0a-407d-8701-06691bcd46c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j9b2" Sep 08 18:23:40.427849 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.427667 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5csb7\" (UniqueName: \"kubernetes.io/projected/e5d99ca2-78e9-43bb-946d-a944ffd92d62-kube-api-access-5csb7\") pod \"node-ca-dwsrv\" (UID: \"e5d99ca2-78e9-43bb-946d-a944ffd92d62\") " pod="openshift-image-registry/node-ca-dwsrv" Sep 08 18:23:40.427849 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.427694 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7h8f\" (UniqueName: \"kubernetes.io/projected/f5b51dfb-01a9-4412-bd5e-8a84b94035d6-kube-api-access-v7h8f\") pod \"iptables-alerter-kqt8b\" (UID: \"f5b51dfb-01a9-4412-bd5e-8a84b94035d6\") " pod="openshift-network-operator/iptables-alerter-kqt8b" Sep 08 18:23:40.427849 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.427683 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e5d99ca2-78e9-43bb-946d-a944ffd92d62-host\") pod \"node-ca-dwsrv\" (UID: \"e5d99ca2-78e9-43bb-946d-a944ffd92d62\") " pod="openshift-image-registry/node-ca-dwsrv" Sep 08 18:23:40.427849 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.427732 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-etc-openvswitch\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.427849 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.427749 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-run-openvswitch\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.427849 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.427739 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a9136afd-27d0-4789-925d-0f7934e8f79b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vw886\" (UID: \"a9136afd-27d0-4789-925d-0f7934e8f79b\") " pod="openshift-multus/multus-additional-cni-plugins-vw886" Sep 08 18:23:40.427849 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.427824 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a9136afd-27d0-4789-925d-0f7934e8f79b-cni-binary-copy\") pod \"multus-additional-cni-plugins-vw886\" (UID: \"a9136afd-27d0-4789-925d-0f7934e8f79b\") " pod="openshift-multus/multus-additional-cni-plugins-vw886" Sep 08 18:23:40.427849 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.427856 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-host-run-ovn-kubernetes\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.428414 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.427859 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a9136afd-27d0-4789-925d-0f7934e8f79b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vw886\" (UID: \"a9136afd-27d0-4789-925d-0f7934e8f79b\") " pod="openshift-multus/multus-additional-cni-plugins-vw886" Sep 08 18:23:40.428414 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.427864 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4e8f614b-fe0a-407d-8701-06691bcd46c7-socket-dir\") pod \"aws-ebs-csi-driver-node-2j9b2\" (UID: \"4e8f614b-fe0a-407d-8701-06691bcd46c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j9b2" Sep 08 18:23:40.428414 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.427910 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-ovn-node-metrics-cert\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.428414 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.427958 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mdvx\" (UniqueName: \"kubernetes.io/projected/deed349e-43fc-4702-aef1-b9334795fd8e-kube-api-access-9mdvx\") pod \"network-check-target-tqthm\" (UID: \"deed349e-43fc-4702-aef1-b9334795fd8e\") " pod="openshift-network-diagnostics/network-check-target-tqthm" Sep 08 18:23:40.428414 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.427986 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-host-run-netns\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.428414 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.428010 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-run-ovn\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.428414 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.428037 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-ovnkube-script-lib\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.428414 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.428063 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cfadca84-d487-4e8d-8c7c-12369bab945d-konnectivity-ca\") pod \"konnectivity-agent-29snt\" (UID: \"cfadca84-d487-4e8d-8c7c-12369bab945d\") " pod="kube-system/konnectivity-agent-29snt" Sep 08 18:23:40.428414 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.428096 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-node-log\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.428414 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.428126 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxps6\" (UniqueName: \"kubernetes.io/projected/4e8f614b-fe0a-407d-8701-06691bcd46c7-kube-api-access-qxps6\") pod \"aws-ebs-csi-driver-node-2j9b2\" (UID: \"4e8f614b-fe0a-407d-8701-06691bcd46c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j9b2" Sep 08 18:23:40.428414 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.428151 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-run-systemd\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.428414 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.428162 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-host-run-ovn-kubernetes\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.428414 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.428175 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4e8f614b-fe0a-407d-8701-06691bcd46c7-registration-dir\") pod \"aws-ebs-csi-driver-node-2j9b2\" (UID: \"4e8f614b-fe0a-407d-8701-06691bcd46c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j9b2" Sep 08 18:23:40.428414 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.428202 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4e8f614b-fe0a-407d-8701-06691bcd46c7-etc-selinux\") pod \"aws-ebs-csi-driver-node-2j9b2\" (UID: \"4e8f614b-fe0a-407d-8701-06691bcd46c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j9b2" Sep 08 18:23:40.428414 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.428226 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e5d99ca2-78e9-43bb-946d-a944ffd92d62-serviceca\") pod \"node-ca-dwsrv\" (UID: \"e5d99ca2-78e9-43bb-946d-a944ffd92d62\") " pod="openshift-image-registry/node-ca-dwsrv" Sep 08 18:23:40.428414 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.428257 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f5b51dfb-01a9-4412-bd5e-8a84b94035d6-iptables-alerter-script\") pod \"iptables-alerter-kqt8b\" (UID: \"f5b51dfb-01a9-4412-bd5e-8a84b94035d6\") " pod="openshift-network-operator/iptables-alerter-kqt8b" Sep 08 18:23:40.428414 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.428280 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-log-socket\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.429362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.428308 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjg9s\" (UniqueName: \"kubernetes.io/projected/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-kube-api-access-kjg9s\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.429362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.428335 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4e8f614b-fe0a-407d-8701-06691bcd46c7-sys-fs\") pod \"aws-ebs-csi-driver-node-2j9b2\" (UID: \"4e8f614b-fe0a-407d-8701-06691bcd46c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j9b2" Sep 08 18:23:40.429362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.428367 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4e8f614b-fe0a-407d-8701-06691bcd46c7-device-dir\") pod \"aws-ebs-csi-driver-node-2j9b2\" (UID: \"4e8f614b-fe0a-407d-8701-06691bcd46c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j9b2" Sep 08 18:23:40.429362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.428396 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f5b51dfb-01a9-4412-bd5e-8a84b94035d6-host-slash\") pod \"iptables-alerter-kqt8b\" (UID: \"f5b51dfb-01a9-4412-bd5e-8a84b94035d6\") " pod="openshift-network-operator/iptables-alerter-kqt8b" Sep 08 18:23:40.429362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.428424 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9136afd-27d0-4789-925d-0f7934e8f79b-system-cni-dir\") pod \"multus-additional-cni-plugins-vw886\" (UID: \"a9136afd-27d0-4789-925d-0f7934e8f79b\") " pod="openshift-multus/multus-additional-cni-plugins-vw886" Sep 08 18:23:40.429362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.428448 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cfadca84-d487-4e8d-8c7c-12369bab945d-agent-certs\") pod \"konnectivity-agent-29snt\" (UID: \"cfadca84-d487-4e8d-8c7c-12369bab945d\") " pod="kube-system/konnectivity-agent-29snt" Sep 08 18:23:40.429362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.428473 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-host-kubelet\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.429362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.428500 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4e8f614b-fe0a-407d-8701-06691bcd46c7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2j9b2\" (UID: \"4e8f614b-fe0a-407d-8701-06691bcd46c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j9b2" Sep 08 18:23:40.429362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.428531 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-host-slash\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.429362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.428556 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-env-overrides\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.429362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.428591 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a9136afd-27d0-4789-925d-0f7934e8f79b-cnibin\") pod \"multus-additional-cni-plugins-vw886\" (UID: \"a9136afd-27d0-4789-925d-0f7934e8f79b\") " pod="openshift-multus/multus-additional-cni-plugins-vw886" Sep 08 18:23:40.429362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.428618 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a9136afd-27d0-4789-925d-0f7934e8f79b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vw886\" (UID: \"a9136afd-27d0-4789-925d-0f7934e8f79b\") " pod="openshift-multus/multus-additional-cni-plugins-vw886" Sep 08 18:23:40.429362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.428647 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slhhp\" (UniqueName: \"kubernetes.io/projected/a9136afd-27d0-4789-925d-0f7934e8f79b-kube-api-access-slhhp\") pod \"multus-additional-cni-plugins-vw886\" (UID: \"a9136afd-27d0-4789-925d-0f7934e8f79b\") " pod="openshift-multus/multus-additional-cni-plugins-vw886" Sep 08 18:23:40.429362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.428673 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-var-lib-openvswitch\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.429362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.428701 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-systemd-units\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.429362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.428774 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-ovnkube-script-lib\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.429362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.428762 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-host-cni-netd\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.430241 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.428831 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.430241 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.428863 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-ovnkube-config\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.430241 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.428897 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-host-cni-bin\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.430241 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.428989 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-host-cni-bin\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.430241 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.429016 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-host-run-netns\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.430241 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.429062 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-run-ovn\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.430241 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.429109 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9136afd-27d0-4789-925d-0f7934e8f79b-system-cni-dir\") pod \"multus-additional-cni-plugins-vw886\" (UID: \"a9136afd-27d0-4789-925d-0f7934e8f79b\") " pod="openshift-multus/multus-additional-cni-plugins-vw886" Sep 08 18:23:40.430241 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.429148 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-node-log\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.430241 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.429173 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a9136afd-27d0-4789-925d-0f7934e8f79b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vw886\" (UID: \"a9136afd-27d0-4789-925d-0f7934e8f79b\") " pod="openshift-multus/multus-additional-cni-plugins-vw886" Sep 08 18:23:40.430241 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.429344 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-run-systemd\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.430241 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.429391 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4e8f614b-fe0a-407d-8701-06691bcd46c7-registration-dir\") pod \"aws-ebs-csi-driver-node-2j9b2\" (UID: \"4e8f614b-fe0a-407d-8701-06691bcd46c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j9b2" Sep 08 18:23:40.430241 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.429472 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4e8f614b-fe0a-407d-8701-06691bcd46c7-etc-selinux\") pod \"aws-ebs-csi-driver-node-2j9b2\" (UID: \"4e8f614b-fe0a-407d-8701-06691bcd46c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j9b2" Sep 08 18:23:40.430241 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.429534 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cfadca84-d487-4e8d-8c7c-12369bab945d-konnectivity-ca\") pod \"konnectivity-agent-29snt\" (UID: \"cfadca84-d487-4e8d-8c7c-12369bab945d\") " pod="kube-system/konnectivity-agent-29snt" Sep 08 18:23:40.430241 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.429650 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a9136afd-27d0-4789-925d-0f7934e8f79b-cni-binary-copy\") pod \"multus-additional-cni-plugins-vw886\" (UID: \"a9136afd-27d0-4789-925d-0f7934e8f79b\") " pod="openshift-multus/multus-additional-cni-plugins-vw886" Sep 08 18:23:40.430241 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.429815 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e5d99ca2-78e9-43bb-946d-a944ffd92d62-serviceca\") pod \"node-ca-dwsrv\" (UID: \"e5d99ca2-78e9-43bb-946d-a944ffd92d62\") " pod="openshift-image-registry/node-ca-dwsrv" Sep 08 18:23:40.430241 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.429847 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-var-lib-openvswitch\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.430241 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.429889 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4e8f614b-fe0a-407d-8701-06691bcd46c7-sys-fs\") pod \"aws-ebs-csi-driver-node-2j9b2\" (UID: \"4e8f614b-fe0a-407d-8701-06691bcd46c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j9b2" Sep 08 18:23:40.431138 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.429901 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-systemd-units\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.431138 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.429946 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4e8f614b-fe0a-407d-8701-06691bcd46c7-device-dir\") pod \"aws-ebs-csi-driver-node-2j9b2\" (UID: \"4e8f614b-fe0a-407d-8701-06691bcd46c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j9b2" Sep 08 18:23:40.431138 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.429970 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-host-cni-netd\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.431138 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.429990 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f5b51dfb-01a9-4412-bd5e-8a84b94035d6-host-slash\") pod \"iptables-alerter-kqt8b\" (UID: \"f5b51dfb-01a9-4412-bd5e-8a84b94035d6\") " pod="openshift-network-operator/iptables-alerter-kqt8b" Sep 08 18:23:40.431138 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.430033 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.431138 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.430042 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4e8f614b-fe0a-407d-8701-06691bcd46c7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2j9b2\" (UID: \"4e8f614b-fe0a-407d-8701-06691bcd46c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j9b2" Sep 08 18:23:40.431138 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.430093 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-host-kubelet\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.431138 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.430131 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-log-socket\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.431138 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.430438 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f5b51dfb-01a9-4412-bd5e-8a84b94035d6-iptables-alerter-script\") pod \"iptables-alerter-kqt8b\" (UID: \"f5b51dfb-01a9-4412-bd5e-8a84b94035d6\") " pod="openshift-network-operator/iptables-alerter-kqt8b" Sep 08 18:23:40.431138 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.430507 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-host-slash\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.431138 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.430558 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a9136afd-27d0-4789-925d-0f7934e8f79b-cnibin\") pod \"multus-additional-cni-plugins-vw886\" (UID: \"a9136afd-27d0-4789-925d-0f7934e8f79b\") " pod="openshift-multus/multus-additional-cni-plugins-vw886" Sep 08 18:23:40.431138 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.430562 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-env-overrides\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.431138 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.430593 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-ovnkube-config\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.431138 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.430989 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a9136afd-27d0-4789-925d-0f7934e8f79b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vw886\" (UID: \"a9136afd-27d0-4789-925d-0f7934e8f79b\") " pod="openshift-multus/multus-additional-cni-plugins-vw886" Sep 08 18:23:40.433107 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.433081 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-ovn-node-metrics-cert\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.433667 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.433624 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cfadca84-d487-4e8d-8c7c-12369bab945d-agent-certs\") pod \"konnectivity-agent-29snt\" (UID: \"cfadca84-d487-4e8d-8c7c-12369bab945d\") " pod="kube-system/konnectivity-agent-29snt" Sep 08 18:23:40.434888 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:40.434866 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 18:23:40.434985 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:40.434894 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 18:23:40.434985 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:40.434908 2555 projected.go:196] Error preparing data for projected volume kube-api-access-9mdvx for pod openshift-network-diagnostics/network-check-target-tqthm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 18:23:40.435096 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:40.434979 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/deed349e-43fc-4702-aef1-b9334795fd8e-kube-api-access-9mdvx podName:deed349e-43fc-4702-aef1-b9334795fd8e nodeName:}" failed. No retries permitted until 2026-09-08 18:23:40.934959831 +0000 UTC m=+3.140945218 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9mdvx" (UniqueName: "kubernetes.io/projected/deed349e-43fc-4702-aef1-b9334795fd8e-kube-api-access-9mdvx") pod "network-check-target-tqthm" (UID: "deed349e-43fc-4702-aef1-b9334795fd8e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 18:23:40.437439 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.437418 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5csb7\" (UniqueName: \"kubernetes.io/projected/e5d99ca2-78e9-43bb-946d-a944ffd92d62-kube-api-access-5csb7\") pod \"node-ca-dwsrv\" (UID: \"e5d99ca2-78e9-43bb-946d-a944ffd92d62\") " pod="openshift-image-registry/node-ca-dwsrv" Sep 08 18:23:40.437812 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.437742 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7h8f\" (UniqueName: \"kubernetes.io/projected/f5b51dfb-01a9-4412-bd5e-8a84b94035d6-kube-api-access-v7h8f\") pod \"iptables-alerter-kqt8b\" (UID: \"f5b51dfb-01a9-4412-bd5e-8a84b94035d6\") " pod="openshift-network-operator/iptables-alerter-kqt8b" Sep 08 18:23:40.441922 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.441582 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxps6\" (UniqueName: \"kubernetes.io/projected/4e8f614b-fe0a-407d-8701-06691bcd46c7-kube-api-access-qxps6\") pod \"aws-ebs-csi-driver-node-2j9b2\" (UID: \"4e8f614b-fe0a-407d-8701-06691bcd46c7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j9b2" Sep 08 18:23:40.441922 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.441831 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slhhp\" (UniqueName: \"kubernetes.io/projected/a9136afd-27d0-4789-925d-0f7934e8f79b-kube-api-access-slhhp\") pod \"multus-additional-cni-plugins-vw886\" (UID: \"a9136afd-27d0-4789-925d-0f7934e8f79b\") " pod="openshift-multus/multus-additional-cni-plugins-vw886" Sep 08 18:23:40.444963 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.444918 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjg9s\" (UniqueName: \"kubernetes.io/projected/0e8f563e-f6b0-4294-af41-3c62f5e8ec6e-kube-api-access-kjg9s\") pod \"ovnkube-node-cc9pd\" (UID: \"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.513914 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.513875 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" Sep 08 18:23:40.520067 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.519890 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lnhcj" Sep 08 18:23:40.530686 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:40.530034 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20991999_23ff_42a8_b754_8c3d3e0866ab.slice/crio-02b014da259fe38f83408ee2b5611c8f88f74290ef24055fc6ddd48d3df170d5 WatchSource:0}: Error finding container 02b014da259fe38f83408ee2b5611c8f88f74290ef24055fc6ddd48d3df170d5: Status 404 returned error can't find the container with id 02b014da259fe38f83408ee2b5611c8f88f74290ef24055fc6ddd48d3df170d5 Sep 08 18:23:40.530880 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.530859 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8sww7" Sep 08 18:23:40.532244 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:40.532220 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod582b7b23_efed_41a0_a58f_726efcdff3a8.slice/crio-37b27cf7e1f2628d3d7718e45e9d2dacf4d5699d1813093f76c2e0aa66889825 WatchSource:0}: Error finding container 37b27cf7e1f2628d3d7718e45e9d2dacf4d5699d1813093f76c2e0aa66889825: Status 404 returned error can't find the container with id 37b27cf7e1f2628d3d7718e45e9d2dacf4d5699d1813093f76c2e0aa66889825 Sep 08 18:23:40.538625 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.538603 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vw886" Sep 08 18:23:40.544808 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:40.544776 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddedc0ee5_f020_456a_988e_7e4833ed34a6.slice/crio-832772c4fc7894f421c3a3715214a06e119bd16c22db6bcc634b4cce490f15d1 WatchSource:0}: Error finding container 832772c4fc7894f421c3a3715214a06e119bd16c22db6bcc634b4cce490f15d1: Status 404 returned error can't find the container with id 832772c4fc7894f421c3a3715214a06e119bd16c22db6bcc634b4cce490f15d1 Sep 08 18:23:40.550143 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.550119 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kqt8b" Sep 08 18:23:40.554778 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:40.554741 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9136afd_27d0_4789_925d_0f7934e8f79b.slice/crio-041d3d7e658c1b7f3a1393c00d181bf2409df67f92318b1594d9f817ea743e67 WatchSource:0}: Error finding container 041d3d7e658c1b7f3a1393c00d181bf2409df67f92318b1594d9f817ea743e67: Status 404 returned error can't find the container with id 041d3d7e658c1b7f3a1393c00d181bf2409df67f92318b1594d9f817ea743e67 Sep 08 18:23:40.557205 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.557045 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-29snt" Sep 08 18:23:40.566076 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.565908 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:23:40.570838 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:40.570811 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfadca84_d487_4e8d_8c7c_12369bab945d.slice/crio-66bbb885eec896944b86a966fd466aa08bc74d0cfbd7b67ec6d30847ea7857ee WatchSource:0}: Error finding container 66bbb885eec896944b86a966fd466aa08bc74d0cfbd7b67ec6d30847ea7857ee: Status 404 returned error can't find the container with id 66bbb885eec896944b86a966fd466aa08bc74d0cfbd7b67ec6d30847ea7857ee Sep 08 18:23:40.574496 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.574470 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j9b2" Sep 08 18:23:40.578686 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:40.578505 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e8f563e_f6b0_4294_af41_3c62f5e8ec6e.slice/crio-09cb03081c280f64f2c54fba2fb1ca1712275cba9a4251473cbddb307f0ca07e WatchSource:0}: Error finding container 09cb03081c280f64f2c54fba2fb1ca1712275cba9a4251473cbddb307f0ca07e: Status 404 returned error can't find the container with id 09cb03081c280f64f2c54fba2fb1ca1712275cba9a4251473cbddb307f0ca07e Sep 08 18:23:40.581358 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.581149 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dwsrv" Sep 08 18:23:40.586458 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:23:40.586433 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e8f614b_fe0a_407d_8701_06691bcd46c7.slice/crio-c423ed3182346f43f4265895b86d2837b8337c7fcb8fc65b34ed61d8edbc8a7b WatchSource:0}: Error finding container c423ed3182346f43f4265895b86d2837b8337c7fcb8fc65b34ed61d8edbc8a7b: Status 404 returned error can't find the container with id c423ed3182346f43f4265895b86d2837b8337c7fcb8fc65b34ed61d8edbc8a7b Sep 08 18:23:40.832804 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.832705 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2744e789-a69b-4c63-b2db-2a7676c746ca-original-pull-secret\") pod \"global-pull-secret-syncer-qbksq\" (UID: \"2744e789-a69b-4c63-b2db-2a7676c746ca\") " pod="kube-system/global-pull-secret-syncer-qbksq" Sep 08 18:23:40.832804 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:40.832761 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2-metrics-certs\") pod \"network-metrics-daemon-qz9vk\" (UID: \"3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2\") " pod="openshift-multus/network-metrics-daemon-qz9vk" Sep 08 18:23:40.833010 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:40.832886 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 18:23:40.833010 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:40.832888 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 18:23:40.833010 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:40.832947 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2-metrics-certs podName:3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2 nodeName:}" failed. No retries permitted until 2026-09-08 18:23:41.832933842 +0000 UTC m=+4.038919219 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2-metrics-certs") pod "network-metrics-daemon-qz9vk" (UID: "3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 18:23:40.833010 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:40.832961 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2744e789-a69b-4c63-b2db-2a7676c746ca-original-pull-secret podName:2744e789-a69b-4c63-b2db-2a7676c746ca nodeName:}" failed. No retries permitted until 2026-09-08 18:23:41.832954924 +0000 UTC m=+4.038940302 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2744e789-a69b-4c63-b2db-2a7676c746ca-original-pull-secret") pod "global-pull-secret-syncer-qbksq" (UID: "2744e789-a69b-4c63-b2db-2a7676c746ca") : object "kube-system"/"original-pull-secret" not registered Sep 08 18:23:41.034598 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:41.034554 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mdvx\" (UniqueName: \"kubernetes.io/projected/deed349e-43fc-4702-aef1-b9334795fd8e-kube-api-access-9mdvx\") pod \"network-check-target-tqthm\" (UID: \"deed349e-43fc-4702-aef1-b9334795fd8e\") " pod="openshift-network-diagnostics/network-check-target-tqthm" Sep 08 18:23:41.034803 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:41.034754 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 18:23:41.034803 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:41.034780 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 18:23:41.034803 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:41.034793 2555 projected.go:196] Error preparing data for projected volume kube-api-access-9mdvx for pod openshift-network-diagnostics/network-check-target-tqthm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 18:23:41.034975 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:41.034864 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/deed349e-43fc-4702-aef1-b9334795fd8e-kube-api-access-9mdvx podName:deed349e-43fc-4702-aef1-b9334795fd8e nodeName:}" failed. No retries permitted until 2026-09-08 18:23:42.034843886 +0000 UTC m=+4.240829265 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9mdvx" (UniqueName: "kubernetes.io/projected/deed349e-43fc-4702-aef1-b9334795fd8e-kube-api-access-9mdvx") pod "network-check-target-tqthm" (UID: "deed349e-43fc-4702-aef1-b9334795fd8e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 18:23:41.259146 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:41.259035 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 18:18:40 +0000 UTC" deadline="2028-02-26 19:36:48.417045461 +0000 UTC" Sep 08 18:23:41.259146 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:41.259079 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12865h13m7.157971717s" Sep 08 18:23:41.318218 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:41.318181 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbksq" Sep 08 18:23:41.318388 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:41.318355 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbksq" podUID="2744e789-a69b-4c63-b2db-2a7676c746ca" Sep 08 18:23:41.332996 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:41.332928 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lnhcj" event={"ID":"582b7b23-efed-41a0-a58f-726efcdff3a8","Type":"ContainerStarted","Data":"37b27cf7e1f2628d3d7718e45e9d2dacf4d5699d1813093f76c2e0aa66889825"} Sep 08 18:23:41.335077 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:41.335038 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" event={"ID":"20991999-23ff-42a8-b754-8c3d3e0866ab","Type":"ContainerStarted","Data":"02b014da259fe38f83408ee2b5611c8f88f74290ef24055fc6ddd48d3df170d5"} Sep 08 18:23:41.336739 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:41.336702 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dwsrv" event={"ID":"e5d99ca2-78e9-43bb-946d-a944ffd92d62","Type":"ContainerStarted","Data":"f466acc58c216a38dd4709a5889fdbd77d988fe6acce44babae9bf903b131158"} Sep 08 18:23:41.338623 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:41.338584 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw886" event={"ID":"a9136afd-27d0-4789-925d-0f7934e8f79b","Type":"ContainerStarted","Data":"041d3d7e658c1b7f3a1393c00d181bf2409df67f92318b1594d9f817ea743e67"} Sep 08 18:23:41.340296 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:41.340220 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j9b2" event={"ID":"4e8f614b-fe0a-407d-8701-06691bcd46c7","Type":"ContainerStarted","Data":"c423ed3182346f43f4265895b86d2837b8337c7fcb8fc65b34ed61d8edbc8a7b"} Sep 08 18:23:41.341922 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:41.341841 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" event={"ID":"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e","Type":"ContainerStarted","Data":"09cb03081c280f64f2c54fba2fb1ca1712275cba9a4251473cbddb307f0ca07e"} Sep 08 18:23:41.343305 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:41.343276 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-29snt" event={"ID":"cfadca84-d487-4e8d-8c7c-12369bab945d","Type":"ContainerStarted","Data":"66bbb885eec896944b86a966fd466aa08bc74d0cfbd7b67ec6d30847ea7857ee"} Sep 08 18:23:41.344541 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:41.344517 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kqt8b" event={"ID":"f5b51dfb-01a9-4412-bd5e-8a84b94035d6","Type":"ContainerStarted","Data":"0228858d52a4d6f7965637f17df62a8a9fe9865e1d16f733d0ce6e6a68f98225"} Sep 08 18:23:41.345937 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:41.345913 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8sww7" event={"ID":"dedc0ee5-f020-456a-988e-7e4833ed34a6","Type":"ContainerStarted","Data":"832772c4fc7894f421c3a3715214a06e119bd16c22db6bcc634b4cce490f15d1"} Sep 08 18:23:41.842759 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:41.842438 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2744e789-a69b-4c63-b2db-2a7676c746ca-original-pull-secret\") pod \"global-pull-secret-syncer-qbksq\" (UID: \"2744e789-a69b-4c63-b2db-2a7676c746ca\") " pod="kube-system/global-pull-secret-syncer-qbksq" Sep 08 18:23:41.842759 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:41.842509 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2-metrics-certs\") pod \"network-metrics-daemon-qz9vk\" (UID: \"3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2\") " pod="openshift-multus/network-metrics-daemon-qz9vk" Sep 08 18:23:41.842759 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:41.842606 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 18:23:41.842759 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:41.842653 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 18:23:41.842759 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:41.842676 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2744e789-a69b-4c63-b2db-2a7676c746ca-original-pull-secret podName:2744e789-a69b-4c63-b2db-2a7676c746ca nodeName:}" failed. No retries permitted until 2026-09-08 18:23:43.842660699 +0000 UTC m=+6.048646077 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2744e789-a69b-4c63-b2db-2a7676c746ca-original-pull-secret") pod "global-pull-secret-syncer-qbksq" (UID: "2744e789-a69b-4c63-b2db-2a7676c746ca") : object "kube-system"/"original-pull-secret" not registered Sep 08 18:23:41.842759 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:41.842710 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2-metrics-certs podName:3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2 nodeName:}" failed. No retries permitted until 2026-09-08 18:23:43.842692214 +0000 UTC m=+6.048677611 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2-metrics-certs") pod "network-metrics-daemon-qz9vk" (UID: "3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 18:23:42.044897 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:42.044856 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mdvx\" (UniqueName: \"kubernetes.io/projected/deed349e-43fc-4702-aef1-b9334795fd8e-kube-api-access-9mdvx\") pod \"network-check-target-tqthm\" (UID: \"deed349e-43fc-4702-aef1-b9334795fd8e\") " pod="openshift-network-diagnostics/network-check-target-tqthm" Sep 08 18:23:42.045077 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:42.045025 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 18:23:42.045077 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:42.045046 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 18:23:42.045077 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:42.045059 2555 projected.go:196] Error preparing data for projected volume kube-api-access-9mdvx for pod openshift-network-diagnostics/network-check-target-tqthm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 18:23:42.045247 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:42.045121 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/deed349e-43fc-4702-aef1-b9334795fd8e-kube-api-access-9mdvx podName:deed349e-43fc-4702-aef1-b9334795fd8e nodeName:}" failed. No retries permitted until 2026-09-08 18:23:44.045102029 +0000 UTC m=+6.251087429 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-9mdvx" (UniqueName: "kubernetes.io/projected/deed349e-43fc-4702-aef1-b9334795fd8e-kube-api-access-9mdvx") pod "network-check-target-tqthm" (UID: "deed349e-43fc-4702-aef1-b9334795fd8e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 18:23:42.260144 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:42.260027 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 18:18:40 +0000 UTC" deadline="2028-03-02 09:19:26.980345632 +0000 UTC" Sep 08 18:23:42.260144 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:42.260095 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12974h55m44.720255141s" Sep 08 18:23:42.318215 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:42.318179 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qz9vk" Sep 08 18:23:42.318364 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:42.318337 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qz9vk" podUID="3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2" Sep 08 18:23:42.318782 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:42.318744 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqthm" Sep 08 18:23:42.318898 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:42.318873 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqthm" podUID="deed349e-43fc-4702-aef1-b9334795fd8e" Sep 08 18:23:43.318517 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:43.318479 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbksq" Sep 08 18:23:43.319371 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:43.318691 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbksq" podUID="2744e789-a69b-4c63-b2db-2a7676c746ca" Sep 08 18:23:43.375501 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:43.374580 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-112.ec2.internal" event={"ID":"5cd5a5d22e051259fc25dc038d5ba1fe","Type":"ContainerStarted","Data":"5393a78e3376b5987c54e737346fcd32755cbae7c9815a083f1f7100e193dcb9"} Sep 08 18:23:43.379660 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:43.379622 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-112.ec2.internal" event={"ID":"c532783d472be20cfe3b62023aa174a5","Type":"ContainerStarted","Data":"417e6098f11b012a84d49066fedde1fde4d36bff3702c7eead7503ba860ea51a"} Sep 08 18:23:43.406583 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:43.406510 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-112.ec2.internal" podStartSLOduration=3.406486791 podStartE2EDuration="3.406486791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 18:23:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:23:43.406016701 +0000 UTC m=+5.612002103" watchObservedRunningTime="2026-09-08 18:23:43.406486791 +0000 UTC m=+5.612472195" Sep 08 18:23:43.860605 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:43.859747 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2744e789-a69b-4c63-b2db-2a7676c746ca-original-pull-secret\") pod \"global-pull-secret-syncer-qbksq\" (UID: \"2744e789-a69b-4c63-b2db-2a7676c746ca\") " pod="kube-system/global-pull-secret-syncer-qbksq" Sep 08 18:23:43.860605 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:43.859817 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2-metrics-certs\") pod \"network-metrics-daemon-qz9vk\" (UID: \"3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2\") " pod="openshift-multus/network-metrics-daemon-qz9vk" Sep 08 18:23:43.860605 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:43.859974 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 18:23:43.860605 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:43.860043 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2-metrics-certs podName:3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2 nodeName:}" failed. No retries permitted until 2026-09-08 18:23:47.860023169 +0000 UTC m=+10.066008553 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2-metrics-certs") pod "network-metrics-daemon-qz9vk" (UID: "3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 18:23:43.860605 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:43.860459 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 18:23:43.860605 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:43.860511 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2744e789-a69b-4c63-b2db-2a7676c746ca-original-pull-secret podName:2744e789-a69b-4c63-b2db-2a7676c746ca nodeName:}" failed. No retries permitted until 2026-09-08 18:23:47.860495209 +0000 UTC m=+10.066480592 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2744e789-a69b-4c63-b2db-2a7676c746ca-original-pull-secret") pod "global-pull-secret-syncer-qbksq" (UID: "2744e789-a69b-4c63-b2db-2a7676c746ca") : object "kube-system"/"original-pull-secret" not registered Sep 08 18:23:44.061878 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:44.061830 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mdvx\" (UniqueName: \"kubernetes.io/projected/deed349e-43fc-4702-aef1-b9334795fd8e-kube-api-access-9mdvx\") pod \"network-check-target-tqthm\" (UID: \"deed349e-43fc-4702-aef1-b9334795fd8e\") " pod="openshift-network-diagnostics/network-check-target-tqthm" Sep 08 18:23:44.062082 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:44.062061 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 18:23:44.062082 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:44.062083 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 18:23:44.062215 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:44.062093 2555 projected.go:196] Error preparing data for projected volume kube-api-access-9mdvx for pod openshift-network-diagnostics/network-check-target-tqthm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 18:23:44.062215 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:44.062140 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/deed349e-43fc-4702-aef1-b9334795fd8e-kube-api-access-9mdvx podName:deed349e-43fc-4702-aef1-b9334795fd8e nodeName:}" failed. No retries permitted until 2026-09-08 18:23:48.062126849 +0000 UTC m=+10.268112241 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-9mdvx" (UniqueName: "kubernetes.io/projected/deed349e-43fc-4702-aef1-b9334795fd8e-kube-api-access-9mdvx") pod "network-check-target-tqthm" (UID: "deed349e-43fc-4702-aef1-b9334795fd8e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 18:23:44.319156 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:44.318373 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qz9vk" Sep 08 18:23:44.319156 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:44.318538 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qz9vk" podUID="3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2" Sep 08 18:23:44.319156 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:44.318954 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqthm" Sep 08 18:23:44.319156 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:44.319053 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqthm" podUID="deed349e-43fc-4702-aef1-b9334795fd8e" Sep 08 18:23:45.318475 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:45.318438 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbksq" Sep 08 18:23:45.318687 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:45.318584 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbksq" podUID="2744e789-a69b-4c63-b2db-2a7676c746ca" Sep 08 18:23:46.318744 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:46.318656 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqthm" Sep 08 18:23:46.319155 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:46.318660 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qz9vk" Sep 08 18:23:46.319155 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:46.318827 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqthm" podUID="deed349e-43fc-4702-aef1-b9334795fd8e" Sep 08 18:23:46.319155 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:46.318916 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qz9vk" podUID="3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2" Sep 08 18:23:46.390563 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:46.390523 2555 generic.go:356] "Generic (PLEG): container finished" podID="5cd5a5d22e051259fc25dc038d5ba1fe" containerID="5393a78e3376b5987c54e737346fcd32755cbae7c9815a083f1f7100e193dcb9" exitCode=0 Sep 08 18:23:46.390792 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:46.390578 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-112.ec2.internal" event={"ID":"5cd5a5d22e051259fc25dc038d5ba1fe","Type":"ContainerDied","Data":"5393a78e3376b5987c54e737346fcd32755cbae7c9815a083f1f7100e193dcb9"} Sep 08 18:23:47.318387 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:47.318344 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbksq" Sep 08 18:23:47.318607 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:47.318493 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbksq" podUID="2744e789-a69b-4c63-b2db-2a7676c746ca" Sep 08 18:23:47.896774 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:47.896737 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2744e789-a69b-4c63-b2db-2a7676c746ca-original-pull-secret\") pod \"global-pull-secret-syncer-qbksq\" (UID: \"2744e789-a69b-4c63-b2db-2a7676c746ca\") " pod="kube-system/global-pull-secret-syncer-qbksq" Sep 08 18:23:47.897231 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:47.896798 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2-metrics-certs\") pod \"network-metrics-daemon-qz9vk\" (UID: \"3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2\") " pod="openshift-multus/network-metrics-daemon-qz9vk" Sep 08 18:23:47.897231 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:47.896860 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 18:23:47.897231 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:47.896914 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 18:23:47.897231 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:47.896935 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2744e789-a69b-4c63-b2db-2a7676c746ca-original-pull-secret podName:2744e789-a69b-4c63-b2db-2a7676c746ca nodeName:}" failed. No retries permitted until 2026-09-08 18:23:55.896917347 +0000 UTC m=+18.102902724 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2744e789-a69b-4c63-b2db-2a7676c746ca-original-pull-secret") pod "global-pull-secret-syncer-qbksq" (UID: "2744e789-a69b-4c63-b2db-2a7676c746ca") : object "kube-system"/"original-pull-secret" not registered Sep 08 18:23:47.897231 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:47.896968 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2-metrics-certs podName:3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2 nodeName:}" failed. No retries permitted until 2026-09-08 18:23:55.896954351 +0000 UTC m=+18.102939743 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2-metrics-certs") pod "network-metrics-daemon-qz9vk" (UID: "3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 18:23:48.099237 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:48.099196 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mdvx\" (UniqueName: \"kubernetes.io/projected/deed349e-43fc-4702-aef1-b9334795fd8e-kube-api-access-9mdvx\") pod \"network-check-target-tqthm\" (UID: \"deed349e-43fc-4702-aef1-b9334795fd8e\") " pod="openshift-network-diagnostics/network-check-target-tqthm" Sep 08 18:23:48.099430 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:48.099365 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 18:23:48.099430 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:48.099385 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 18:23:48.099430 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:48.099395 2555 projected.go:196] Error preparing data for projected volume kube-api-access-9mdvx for pod openshift-network-diagnostics/network-check-target-tqthm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 18:23:48.099596 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:48.099456 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/deed349e-43fc-4702-aef1-b9334795fd8e-kube-api-access-9mdvx podName:deed349e-43fc-4702-aef1-b9334795fd8e nodeName:}" failed. No retries permitted until 2026-09-08 18:23:56.099441167 +0000 UTC m=+18.305426546 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-9mdvx" (UniqueName: "kubernetes.io/projected/deed349e-43fc-4702-aef1-b9334795fd8e-kube-api-access-9mdvx") pod "network-check-target-tqthm" (UID: "deed349e-43fc-4702-aef1-b9334795fd8e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 18:23:48.318565 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:48.318479 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qz9vk" Sep 08 18:23:48.318565 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:48.318479 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqthm" Sep 08 18:23:48.319543 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:48.319506 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qz9vk" podUID="3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2" Sep 08 18:23:48.319720 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:48.319653 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqthm" podUID="deed349e-43fc-4702-aef1-b9334795fd8e" Sep 08 18:23:49.318301 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:49.318263 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbksq" Sep 08 18:23:49.318881 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:49.318388 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbksq" podUID="2744e789-a69b-4c63-b2db-2a7676c746ca" Sep 08 18:23:50.318742 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:50.318249 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qz9vk" Sep 08 18:23:50.318742 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:50.318386 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qz9vk" podUID="3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2" Sep 08 18:23:50.319402 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:50.319260 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqthm" Sep 08 18:23:50.319402 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:50.319345 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqthm" podUID="deed349e-43fc-4702-aef1-b9334795fd8e" Sep 08 18:23:51.318351 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:51.318314 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbksq" Sep 08 18:23:51.318528 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:51.318454 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbksq" podUID="2744e789-a69b-4c63-b2db-2a7676c746ca" Sep 08 18:23:52.319668 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:52.319627 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qz9vk" Sep 08 18:23:52.320105 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:52.319630 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqthm" Sep 08 18:23:52.320105 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:52.319806 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qz9vk" podUID="3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2" Sep 08 18:23:52.320105 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:52.319939 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqthm" podUID="deed349e-43fc-4702-aef1-b9334795fd8e" Sep 08 18:23:52.404590 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:52.404554 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lnhcj" event={"ID":"582b7b23-efed-41a0-a58f-726efcdff3a8","Type":"ContainerStarted","Data":"10400ce02e2ee743e6d73e61b54914aadef48b4d340b12827028f2bb883a92fe"} Sep 08 18:23:52.406298 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:52.406273 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" event={"ID":"20991999-23ff-42a8-b754-8c3d3e0866ab","Type":"ContainerStarted","Data":"05317bed76cef2f46cf05aa75266dbfc3b918e997eff76c219a3bc48e2f1dc8b"} Sep 08 18:23:52.408411 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:52.408384 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-112.ec2.internal" event={"ID":"5cd5a5d22e051259fc25dc038d5ba1fe","Type":"ContainerStarted","Data":"ccb41724fdc006228bff2a6ecd4a88588ef865435fbd585e491e52204f298b79"} Sep 08 18:23:52.410033 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:52.410007 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dwsrv" event={"ID":"e5d99ca2-78e9-43bb-946d-a944ffd92d62","Type":"ContainerStarted","Data":"d68111fe91fa4254f885d0a22e98c6b3ed1229b86795dc5bdef7e5ca86d366e5"} Sep 08 18:23:52.412034 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:52.412011 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw886" event={"ID":"a9136afd-27d0-4789-925d-0f7934e8f79b","Type":"ContainerStarted","Data":"199965a1b2a21f5dc54aa90cb44e8659e59fc5875e8959ceff41beb9de282f08"} Sep 08 18:23:52.414183 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:52.414160 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j9b2" event={"ID":"4e8f614b-fe0a-407d-8701-06691bcd46c7","Type":"ContainerStarted","Data":"44e448a14ff06662aa470b4eac9c763873a6a2514ff0c6578f606bed470a969a"} Sep 08 18:23:52.415973 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:52.415950 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-29snt" event={"ID":"cfadca84-d487-4e8d-8c7c-12369bab945d","Type":"ContainerStarted","Data":"27d0fda1876dd2068d4786ecc6781b0bf513bb81955e2b545e62f47a6594c9a6"} Sep 08 18:23:52.418494 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:52.418456 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-lnhcj" podStartSLOduration=2.190134268 podStartE2EDuration="13.418443107s" totalImagesPullingTime="11.228308839s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 18:23:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:23:52.418066112 +0000 UTC m=+14.624051512" watchObservedRunningTime="2026-09-08 18:23:52.418443107 +0000 UTC m=+14.624428504" Sep 08 18:23:52.431093 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:52.431054 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-29snt" podStartSLOduration=2.245473388 podStartE2EDuration="13.431041681s" totalImagesPullingTime="11.185568293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:23:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:23:52.43102541 +0000 UTC m=+14.637010807" watchObservedRunningTime="2026-09-08 18:23:52.431041681 +0000 UTC m=+14.637027078" Sep 08 18:23:52.448050 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:52.447997 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-dwsrv" podStartSLOduration=2.284275874 podStartE2EDuration="13.447979782s" totalImagesPullingTime="11.163703908s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 18:23:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:23:52.447821819 +0000 UTC m=+14.653807219" watchObservedRunningTime="2026-09-08 18:23:52.447979782 +0000 UTC m=+14.653965181" Sep 08 18:23:52.462959 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:52.462910 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zsfvh" podStartSLOduration=2.080284627 podStartE2EDuration="13.462891368s" totalImagesPullingTime="11.382606741s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 18:23:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:23:52.462394519 +0000 UTC m=+14.668379918" watchObservedRunningTime="2026-09-08 18:23:52.462891368 +0000 UTC m=+14.668876768" Sep 08 18:23:52.508859 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:52.508800 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-112.ec2.internal" podStartSLOduration=12.508782199 podStartE2EDuration="12.508782199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 18:23:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:23:52.508592518 +0000 UTC m=+14.714577916" watchObservedRunningTime="2026-09-08 18:23:52.508782199 +0000 UTC m=+14.714767598" Sep 08 18:23:53.317831 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:53.317798 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbksq" Sep 08 18:23:53.318045 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:53.317904 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbksq" podUID="2744e789-a69b-4c63-b2db-2a7676c746ca" Sep 08 18:23:53.419479 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:53.419445 2555 generic.go:356] "Generic (PLEG): container finished" podID="a9136afd-27d0-4789-925d-0f7934e8f79b" containerID="199965a1b2a21f5dc54aa90cb44e8659e59fc5875e8959ceff41beb9de282f08" exitCode=0 Sep 08 18:23:53.419878 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:53.419532 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw886" event={"ID":"a9136afd-27d0-4789-925d-0f7934e8f79b","Type":"ContainerDied","Data":"199965a1b2a21f5dc54aa90cb44e8659e59fc5875e8959ceff41beb9de282f08"} Sep 08 18:23:54.318849 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:54.318812 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qz9vk" Sep 08 18:23:54.319031 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:54.318943 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qz9vk" podUID="3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2" Sep 08 18:23:54.319031 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:54.318992 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqthm" Sep 08 18:23:54.319127 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:54.319109 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqthm" podUID="deed349e-43fc-4702-aef1-b9334795fd8e" Sep 08 18:23:54.425019 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:54.424977 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kqt8b" event={"ID":"f5b51dfb-01a9-4412-bd5e-8a84b94035d6","Type":"ContainerStarted","Data":"52ae89fff47665c7ca1e7aba764f64b2279558e3c03904aa29dba500bd73e4c5"} Sep 08 18:23:54.439222 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:54.439162 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-kqt8b" podStartSLOduration=4.267628153 podStartE2EDuration="15.439140174s" totalImagesPullingTime="11.171512021s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 18:23:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:23:54.438811307 +0000 UTC m=+16.644796720" watchObservedRunningTime="2026-09-08 18:23:54.439140174 +0000 UTC m=+16.645125556" Sep 08 18:23:55.317768 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:55.317738 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbksq" Sep 08 18:23:55.317975 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:55.317841 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbksq" podUID="2744e789-a69b-4c63-b2db-2a7676c746ca" Sep 08 18:23:55.961274 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:55.961237 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2744e789-a69b-4c63-b2db-2a7676c746ca-original-pull-secret\") pod \"global-pull-secret-syncer-qbksq\" (UID: \"2744e789-a69b-4c63-b2db-2a7676c746ca\") " pod="kube-system/global-pull-secret-syncer-qbksq" Sep 08 18:23:55.961733 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:55.961290 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2-metrics-certs\") pod \"network-metrics-daemon-qz9vk\" (UID: \"3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2\") " pod="openshift-multus/network-metrics-daemon-qz9vk" Sep 08 18:23:55.961733 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:55.961393 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 18:23:55.961733 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:55.961395 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 18:23:55.961733 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:55.961465 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2-metrics-certs podName:3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2 nodeName:}" failed. No retries permitted until 2026-09-08 18:24:11.961437935 +0000 UTC m=+34.167423312 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2-metrics-certs") pod "network-metrics-daemon-qz9vk" (UID: "3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 18:23:55.961733 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:55.961484 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2744e789-a69b-4c63-b2db-2a7676c746ca-original-pull-secret podName:2744e789-a69b-4c63-b2db-2a7676c746ca nodeName:}" failed. No retries permitted until 2026-09-08 18:24:11.961474981 +0000 UTC m=+34.167460358 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2744e789-a69b-4c63-b2db-2a7676c746ca-original-pull-secret") pod "global-pull-secret-syncer-qbksq" (UID: "2744e789-a69b-4c63-b2db-2a7676c746ca") : object "kube-system"/"original-pull-secret" not registered Sep 08 18:23:56.163098 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:56.163061 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mdvx\" (UniqueName: \"kubernetes.io/projected/deed349e-43fc-4702-aef1-b9334795fd8e-kube-api-access-9mdvx\") pod \"network-check-target-tqthm\" (UID: \"deed349e-43fc-4702-aef1-b9334795fd8e\") " pod="openshift-network-diagnostics/network-check-target-tqthm" Sep 08 18:23:56.163274 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:56.163246 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 18:23:56.163274 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:56.163274 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 18:23:56.163393 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:56.163289 2555 projected.go:196] Error preparing data for projected volume kube-api-access-9mdvx for pod openshift-network-diagnostics/network-check-target-tqthm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 18:23:56.163393 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:56.163350 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/deed349e-43fc-4702-aef1-b9334795fd8e-kube-api-access-9mdvx podName:deed349e-43fc-4702-aef1-b9334795fd8e nodeName:}" failed. No retries permitted until 2026-09-08 18:24:12.163331552 +0000 UTC m=+34.369316933 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-9mdvx" (UniqueName: "kubernetes.io/projected/deed349e-43fc-4702-aef1-b9334795fd8e-kube-api-access-9mdvx") pod "network-check-target-tqthm" (UID: "deed349e-43fc-4702-aef1-b9334795fd8e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 18:23:56.319141 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:56.318564 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqthm" Sep 08 18:23:56.319141 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:56.318583 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qz9vk" Sep 08 18:23:56.319141 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:56.318729 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqthm" podUID="deed349e-43fc-4702-aef1-b9334795fd8e" Sep 08 18:23:56.319141 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:56.318865 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qz9vk" podUID="3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2" Sep 08 18:23:57.037572 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:57.037520 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-29snt" Sep 08 18:23:57.038398 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:57.038375 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-29snt" Sep 08 18:23:57.317924 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:57.317846 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbksq" Sep 08 18:23:57.318061 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:57.317956 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbksq" podUID="2744e789-a69b-4c63-b2db-2a7676c746ca" Sep 08 18:23:57.432187 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:57.432156 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-29snt" Sep 08 18:23:57.432732 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:57.432708 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-29snt" Sep 08 18:23:58.319291 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:58.319262 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qz9vk" Sep 08 18:23:58.319727 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:58.319349 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqthm" Sep 08 18:23:58.319727 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:58.319379 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qz9vk" podUID="3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2" Sep 08 18:23:58.319727 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:58.319461 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqthm" podUID="deed349e-43fc-4702-aef1-b9334795fd8e" Sep 08 18:23:59.317822 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:59.317780 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbksq" Sep 08 18:23:59.317997 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:23:59.317906 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbksq" podUID="2744e789-a69b-4c63-b2db-2a7676c746ca" Sep 08 18:23:59.945098 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:23:59.944814 2555 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 08 18:24:00.292367 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:00.292244 2555 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-08T18:23:59.945026614Z","UUID":"1c4cc0a4-7423-471c-a5ad-027ece0ca75c","Handler":null,"Name":"","Endpoint":""} Sep 08 18:24:00.294057 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:00.294031 2555 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 08 18:24:00.294195 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:00.294077 2555 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 08 18:24:00.317935 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:00.317907 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qz9vk" Sep 08 18:24:00.318094 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:00.318057 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qz9vk" podUID="3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2" Sep 08 18:24:00.318094 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:00.318075 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqthm" Sep 08 18:24:00.318183 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:00.318166 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqthm" podUID="deed349e-43fc-4702-aef1-b9334795fd8e" Sep 08 18:24:00.441810 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:00.441709 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j9b2" event={"ID":"4e8f614b-fe0a-407d-8701-06691bcd46c7","Type":"ContainerStarted","Data":"5512993384b07ee0986b7c30fdac8d70e5010868e444721f8c57a7767010a610"} Sep 08 18:24:00.445749 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:00.445717 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" event={"ID":"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e","Type":"ContainerStarted","Data":"d5160af6f1c12541d4e90f39bb9779a619dc239901df861561a5e21b667f1285"} Sep 08 18:24:00.445901 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:00.445796 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" event={"ID":"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e","Type":"ContainerStarted","Data":"b7b905df126694c6a3978259a9cc3471260a9b3bab9b19c80a4d3d63bfe14706"} Sep 08 18:24:00.445901 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:00.445839 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" event={"ID":"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e","Type":"ContainerStarted","Data":"ab68c511ee8eef5fdaa058e06404f2d0c479daecf85406dcb7940e91fd3a4e03"} Sep 08 18:24:00.445901 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:00.445877 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" event={"ID":"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e","Type":"ContainerStarted","Data":"af86591c087fe8b1a4b6868fac7244ba57774496914d5f49a3fbf1148d767d9d"} Sep 08 18:24:00.446066 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:00.445918 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" event={"ID":"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e","Type":"ContainerStarted","Data":"6575ebed1fda521748d970f40797fb9a4faf1ef388edf0137d943b68eabfc095"} Sep 08 18:24:00.446066 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:00.445955 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" event={"ID":"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e","Type":"ContainerStarted","Data":"197b5ca570cf4ca0b56f7349dfa724ff8529af0e7286cd6afc738760880532b1"} Sep 08 18:24:00.447410 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:00.447384 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8sww7" event={"ID":"dedc0ee5-f020-456a-988e-7e4833ed34a6","Type":"ContainerStarted","Data":"ebab8280be2666c5a5d586a1d29477fccd89974842203df12d053e32e65f84a8"} Sep 08 18:24:00.465253 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:00.465202 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-8sww7" podStartSLOduration=2.237489426 podStartE2EDuration="21.465187482s" totalImagesPullingTime="19.227698056s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 18:23:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:24:00.464651258 +0000 UTC m=+22.670636667" watchObservedRunningTime="2026-09-08 18:24:00.465187482 +0000 UTC m=+22.671172880" Sep 08 18:24:01.318244 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:01.318217 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbksq" Sep 08 18:24:01.318633 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:01.318356 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbksq" podUID="2744e789-a69b-4c63-b2db-2a7676c746ca" Sep 08 18:24:01.451405 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:01.451364 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j9b2" event={"ID":"4e8f614b-fe0a-407d-8701-06691bcd46c7","Type":"ContainerStarted","Data":"89a8f62c281b9c336d7cc066d8fdb018e798540336572f08becb7cf635b2eb58"} Sep 08 18:24:01.468702 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:01.468641 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2j9b2" podStartSLOduration=2.356554354 podStartE2EDuration="22.46862366s" totalImagesPullingTime="20.112069306s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 18:23:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:24:01.468136246 +0000 UTC m=+23.674121657" watchObservedRunningTime="2026-09-08 18:24:01.46862366 +0000 UTC m=+23.674609102" Sep 08 18:24:02.317862 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:02.317840 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqthm" Sep 08 18:24:02.317862 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:02.317857 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qz9vk" Sep 08 18:24:02.317993 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:02.317972 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqthm" podUID="deed349e-43fc-4702-aef1-b9334795fd8e" Sep 08 18:24:02.318130 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:02.318113 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qz9vk" podUID="3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2" Sep 08 18:24:02.464826 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:02.464782 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" event={"ID":"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e","Type":"ContainerStarted","Data":"37ff2da80d1390e5a0e529d8a8d80aa5f94d4700aeba5bea2cf8599faf72cbf8"} Sep 08 18:24:03.318351 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:03.318314 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbksq" Sep 08 18:24:03.318527 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:03.318424 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbksq" podUID="2744e789-a69b-4c63-b2db-2a7676c746ca" Sep 08 18:24:04.318218 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:04.318179 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqthm" Sep 08 18:24:04.318710 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:04.318316 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqthm" podUID="deed349e-43fc-4702-aef1-b9334795fd8e" Sep 08 18:24:04.318816 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:04.318799 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qz9vk" Sep 08 18:24:04.318923 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:04.318889 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qz9vk" podUID="3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2" Sep 08 18:24:05.318290 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:05.318107 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbksq" Sep 08 18:24:05.318663 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:05.318407 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbksq" podUID="2744e789-a69b-4c63-b2db-2a7676c746ca" Sep 08 18:24:05.473144 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:05.473055 2555 generic.go:356] "Generic (PLEG): container finished" podID="a9136afd-27d0-4789-925d-0f7934e8f79b" containerID="efcd689aa9551cd9d4515226b7b589344a86fcb0f12485935f6b52cbc9293318" exitCode=0 Sep 08 18:24:05.473296 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:05.473144 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw886" event={"ID":"a9136afd-27d0-4789-925d-0f7934e8f79b","Type":"ContainerDied","Data":"efcd689aa9551cd9d4515226b7b589344a86fcb0f12485935f6b52cbc9293318"} Sep 08 18:24:05.477140 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:05.477111 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" event={"ID":"0e8f563e-f6b0-4294-af41-3c62f5e8ec6e","Type":"ContainerStarted","Data":"ff2d858a4b630225250b7c4c3931edc969837a59ecc25b0d06a9036c0dde938d"} Sep 08 18:24:05.477348 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:05.477321 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:24:05.477453 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:05.477399 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:24:05.477494 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:05.477457 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:24:05.493304 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:05.493280 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:24:05.493470 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:05.493376 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:24:05.514550 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:05.514502 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" podStartSLOduration=7.320032716 podStartE2EDuration="26.514485803s" totalImagesPullingTime="19.194453087s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 18:23:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:24:05.513671484 +0000 UTC m=+27.719656881" watchObservedRunningTime="2026-09-08 18:24:05.514485803 +0000 UTC m=+27.720471219" Sep 08 18:24:06.324637 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:06.324601 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qz9vk" Sep 08 18:24:06.325249 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:06.324763 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qz9vk" podUID="3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2" Sep 08 18:24:06.325249 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:06.324842 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqthm" Sep 08 18:24:06.325249 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:06.324907 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqthm" podUID="deed349e-43fc-4702-aef1-b9334795fd8e" Sep 08 18:24:07.280571 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:07.280535 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qz9vk"] Sep 08 18:24:07.280730 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:07.280645 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qz9vk" Sep 08 18:24:07.280794 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:07.280763 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qz9vk" podUID="3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2" Sep 08 18:24:07.283788 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:07.283745 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qbksq"] Sep 08 18:24:07.283941 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:07.283877 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbksq" Sep 08 18:24:07.283941 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:07.283901 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tqthm"] Sep 08 18:24:07.284037 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:07.283967 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqthm" Sep 08 18:24:07.284037 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:07.283975 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbksq" podUID="2744e789-a69b-4c63-b2db-2a7676c746ca" Sep 08 18:24:07.284037 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:07.284028 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqthm" podUID="deed349e-43fc-4702-aef1-b9334795fd8e" Sep 08 18:24:07.483387 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:07.483301 2555 generic.go:356] "Generic (PLEG): container finished" podID="a9136afd-27d0-4789-925d-0f7934e8f79b" containerID="81ff6461419c1085e899c7a7e00fd4d0474572fd2826dd158c69b9b9618d91c9" exitCode=0 Sep 08 18:24:07.484188 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:07.483383 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw886" event={"ID":"a9136afd-27d0-4789-925d-0f7934e8f79b","Type":"ContainerDied","Data":"81ff6461419c1085e899c7a7e00fd4d0474572fd2826dd158c69b9b9618d91c9"} Sep 08 18:24:08.488788 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:08.488677 2555 generic.go:356] "Generic (PLEG): container finished" podID="a9136afd-27d0-4789-925d-0f7934e8f79b" containerID="cb525b521b59b0f42b75e785430bf11bb849dc57df6eebebdefc2a1540b1441a" exitCode=0 Sep 08 18:24:08.488788 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:08.488728 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw886" event={"ID":"a9136afd-27d0-4789-925d-0f7934e8f79b","Type":"ContainerDied","Data":"cb525b521b59b0f42b75e785430bf11bb849dc57df6eebebdefc2a1540b1441a"} Sep 08 18:24:09.318134 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:09.317820 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqthm" Sep 08 18:24:09.318292 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:09.318045 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qz9vk" Sep 08 18:24:09.318292 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:09.318174 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqthm" podUID="deed349e-43fc-4702-aef1-b9334795fd8e" Sep 08 18:24:09.318292 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:09.318191 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbksq" Sep 08 18:24:09.318430 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:09.318299 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qz9vk" podUID="3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2" Sep 08 18:24:09.318430 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:09.318387 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbksq" podUID="2744e789-a69b-4c63-b2db-2a7676c746ca" Sep 08 18:24:11.317939 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:11.317901 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbksq" Sep 08 18:24:11.318366 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:11.317902 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qz9vk" Sep 08 18:24:11.318366 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:11.317934 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqthm" Sep 08 18:24:11.319659 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:11.318866 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qbksq" podUID="2744e789-a69b-4c63-b2db-2a7676c746ca" Sep 08 18:24:11.321025 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:11.319944 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tqthm" podUID="deed349e-43fc-4702-aef1-b9334795fd8e" Sep 08 18:24:11.321164 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:11.321012 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qz9vk" podUID="3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2" Sep 08 18:24:11.977131 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:11.977091 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2744e789-a69b-4c63-b2db-2a7676c746ca-original-pull-secret\") pod \"global-pull-secret-syncer-qbksq\" (UID: \"2744e789-a69b-4c63-b2db-2a7676c746ca\") " pod="kube-system/global-pull-secret-syncer-qbksq" Sep 08 18:24:11.977343 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:11.977145 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2-metrics-certs\") pod \"network-metrics-daemon-qz9vk\" (UID: \"3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2\") " pod="openshift-multus/network-metrics-daemon-qz9vk" Sep 08 18:24:11.977343 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:11.977251 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 08 18:24:11.977343 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:11.977282 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 18:24:11.977343 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:11.977319 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2744e789-a69b-4c63-b2db-2a7676c746ca-original-pull-secret podName:2744e789-a69b-4c63-b2db-2a7676c746ca nodeName:}" failed. No retries permitted until 2026-09-08 18:24:43.977302343 +0000 UTC m=+66.183287719 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2744e789-a69b-4c63-b2db-2a7676c746ca-original-pull-secret") pod "global-pull-secret-syncer-qbksq" (UID: "2744e789-a69b-4c63-b2db-2a7676c746ca") : object "kube-system"/"original-pull-secret" not registered Sep 08 18:24:11.977343 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:11.977340 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2-metrics-certs podName:3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2 nodeName:}" failed. No retries permitted until 2026-09-08 18:24:43.977325289 +0000 UTC m=+66.183310665 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2-metrics-certs") pod "network-metrics-daemon-qz9vk" (UID: "3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 08 18:24:12.179450 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.179412 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mdvx\" (UniqueName: \"kubernetes.io/projected/deed349e-43fc-4702-aef1-b9334795fd8e-kube-api-access-9mdvx\") pod \"network-check-target-tqthm\" (UID: \"deed349e-43fc-4702-aef1-b9334795fd8e\") " pod="openshift-network-diagnostics/network-check-target-tqthm" Sep 08 18:24:12.179671 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:12.179571 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 08 18:24:12.179671 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:12.179588 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 08 18:24:12.179671 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:12.179598 2555 projected.go:196] Error preparing data for projected volume kube-api-access-9mdvx for pod openshift-network-diagnostics/network-check-target-tqthm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 18:24:12.179671 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:12.179658 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/deed349e-43fc-4702-aef1-b9334795fd8e-kube-api-access-9mdvx podName:deed349e-43fc-4702-aef1-b9334795fd8e nodeName:}" failed. No retries permitted until 2026-09-08 18:24:44.179638306 +0000 UTC m=+66.385623683 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-9mdvx" (UniqueName: "kubernetes.io/projected/deed349e-43fc-4702-aef1-b9334795fd8e-kube-api-access-9mdvx") pod "network-check-target-tqthm" (UID: "deed349e-43fc-4702-aef1-b9334795fd8e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 08 18:24:12.612628 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.612591 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-112.ec2.internal" event="NodeReady" Sep 08 18:24:12.613084 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.612770 2555 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 08 18:24:12.647356 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.647325 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6c87577785-px7mf"] Sep 08 18:24:12.662979 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.662951 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c87577785-px7mf" Sep 08 18:24:12.663847 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.663245 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c87577785-px7mf"] Sep 08 18:24:12.665348 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.665215 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 08 18:24:12.665348 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.665338 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 08 18:24:12.665556 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.665453 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-cm2c5\"" Sep 08 18:24:12.665995 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.665822 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 08 18:24:12.668086 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.668055 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-pzzmm"] Sep 08 18:24:12.671716 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.671677 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 08 18:24:12.690923 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.690883 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-k4dxz"] Sep 08 18:24:12.691078 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.691010 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pzzmm" Sep 08 18:24:12.693108 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.693082 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 08 18:24:12.693222 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.693119 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vbqmk\"" Sep 08 18:24:12.693825 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.693803 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 08 18:24:12.706954 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.706920 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pzzmm"] Sep 08 18:24:12.707098 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.706971 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-k4dxz"] Sep 08 18:24:12.707098 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.707061 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-k4dxz" Sep 08 18:24:12.708987 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.708963 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jndxk\"" Sep 08 18:24:12.709104 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.709059 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 08 18:24:12.709167 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.709138 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 08 18:24:12.709296 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.709280 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 08 18:24:12.783312 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.783209 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/612d5222-58c3-4da0-adc4-760f3fcd1843-image-registry-private-configuration\") pod \"image-registry-6c87577785-px7mf\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " pod="openshift-image-registry/image-registry-6c87577785-px7mf" Sep 08 18:24:12.783312 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.783265 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9de9b23a-f1e9-4b44-87a2-680997e061f0-tmp-dir\") pod \"dns-default-pzzmm\" (UID: \"9de9b23a-f1e9-4b44-87a2-680997e061f0\") " pod="openshift-dns/dns-default-pzzmm" Sep 08 18:24:12.783531 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.783331 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2nc7\" (UniqueName: \"kubernetes.io/projected/9de9b23a-f1e9-4b44-87a2-680997e061f0-kube-api-access-j2nc7\") pod \"dns-default-pzzmm\" (UID: \"9de9b23a-f1e9-4b44-87a2-680997e061f0\") " pod="openshift-dns/dns-default-pzzmm" Sep 08 18:24:12.783531 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.783386 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9de9b23a-f1e9-4b44-87a2-680997e061f0-config-volume\") pod \"dns-default-pzzmm\" (UID: \"9de9b23a-f1e9-4b44-87a2-680997e061f0\") " pod="openshift-dns/dns-default-pzzmm" Sep 08 18:24:12.783531 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.783414 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/612d5222-58c3-4da0-adc4-760f3fcd1843-ca-trust-extracted\") pod \"image-registry-6c87577785-px7mf\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " pod="openshift-image-registry/image-registry-6c87577785-px7mf" Sep 08 18:24:12.783531 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.783501 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/612d5222-58c3-4da0-adc4-760f3fcd1843-registry-certificates\") pod \"image-registry-6c87577785-px7mf\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " pod="openshift-image-registry/image-registry-6c87577785-px7mf" Sep 08 18:24:12.783714 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.783553 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/612d5222-58c3-4da0-adc4-760f3fcd1843-trusted-ca\") pod \"image-registry-6c87577785-px7mf\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " pod="openshift-image-registry/image-registry-6c87577785-px7mf" Sep 08 18:24:12.783714 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.783579 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/612d5222-58c3-4da0-adc4-760f3fcd1843-bound-sa-token\") pod \"image-registry-6c87577785-px7mf\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " pod="openshift-image-registry/image-registry-6c87577785-px7mf" Sep 08 18:24:12.783714 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.783641 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9de9b23a-f1e9-4b44-87a2-680997e061f0-metrics-tls\") pod \"dns-default-pzzmm\" (UID: \"9de9b23a-f1e9-4b44-87a2-680997e061f0\") " pod="openshift-dns/dns-default-pzzmm" Sep 08 18:24:12.783714 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.783679 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/612d5222-58c3-4da0-adc4-760f3fcd1843-installation-pull-secrets\") pod \"image-registry-6c87577785-px7mf\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " pod="openshift-image-registry/image-registry-6c87577785-px7mf" Sep 08 18:24:12.783882 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.783724 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a067ecec-0c84-4299-812c-6441d126104f-cert\") pod \"ingress-canary-k4dxz\" (UID: \"a067ecec-0c84-4299-812c-6441d126104f\") " pod="openshift-ingress-canary/ingress-canary-k4dxz" Sep 08 18:24:12.783882 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.783744 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78rp2\" (UniqueName: \"kubernetes.io/projected/612d5222-58c3-4da0-adc4-760f3fcd1843-kube-api-access-78rp2\") pod \"image-registry-6c87577785-px7mf\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " pod="openshift-image-registry/image-registry-6c87577785-px7mf" Sep 08 18:24:12.783882 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.783826 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/612d5222-58c3-4da0-adc4-760f3fcd1843-registry-tls\") pod \"image-registry-6c87577785-px7mf\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " pod="openshift-image-registry/image-registry-6c87577785-px7mf" Sep 08 18:24:12.783882 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.783846 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbkrm\" (UniqueName: \"kubernetes.io/projected/a067ecec-0c84-4299-812c-6441d126104f-kube-api-access-mbkrm\") pod \"ingress-canary-k4dxz\" (UID: \"a067ecec-0c84-4299-812c-6441d126104f\") " pod="openshift-ingress-canary/ingress-canary-k4dxz" Sep 08 18:24:12.884597 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.884554 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9de9b23a-f1e9-4b44-87a2-680997e061f0-config-volume\") pod \"dns-default-pzzmm\" (UID: \"9de9b23a-f1e9-4b44-87a2-680997e061f0\") " pod="openshift-dns/dns-default-pzzmm" Sep 08 18:24:12.884781 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.884611 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/612d5222-58c3-4da0-adc4-760f3fcd1843-ca-trust-extracted\") pod \"image-registry-6c87577785-px7mf\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " pod="openshift-image-registry/image-registry-6c87577785-px7mf" Sep 08 18:24:12.884781 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.884653 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/612d5222-58c3-4da0-adc4-760f3fcd1843-registry-certificates\") pod \"image-registry-6c87577785-px7mf\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " pod="openshift-image-registry/image-registry-6c87577785-px7mf" Sep 08 18:24:12.884781 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.884703 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/612d5222-58c3-4da0-adc4-760f3fcd1843-trusted-ca\") pod \"image-registry-6c87577785-px7mf\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " pod="openshift-image-registry/image-registry-6c87577785-px7mf" Sep 08 18:24:12.884781 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.884731 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/612d5222-58c3-4da0-adc4-760f3fcd1843-bound-sa-token\") pod \"image-registry-6c87577785-px7mf\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " pod="openshift-image-registry/image-registry-6c87577785-px7mf" Sep 08 18:24:12.884781 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.884760 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9de9b23a-f1e9-4b44-87a2-680997e061f0-metrics-tls\") pod \"dns-default-pzzmm\" (UID: \"9de9b23a-f1e9-4b44-87a2-680997e061f0\") " pod="openshift-dns/dns-default-pzzmm" Sep 08 18:24:12.885018 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.884790 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/612d5222-58c3-4da0-adc4-760f3fcd1843-installation-pull-secrets\") pod \"image-registry-6c87577785-px7mf\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " pod="openshift-image-registry/image-registry-6c87577785-px7mf" Sep 08 18:24:12.885018 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.884815 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a067ecec-0c84-4299-812c-6441d126104f-cert\") pod \"ingress-canary-k4dxz\" (UID: \"a067ecec-0c84-4299-812c-6441d126104f\") " pod="openshift-ingress-canary/ingress-canary-k4dxz" Sep 08 18:24:12.885018 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.884844 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78rp2\" (UniqueName: \"kubernetes.io/projected/612d5222-58c3-4da0-adc4-760f3fcd1843-kube-api-access-78rp2\") pod \"image-registry-6c87577785-px7mf\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " pod="openshift-image-registry/image-registry-6c87577785-px7mf" Sep 08 18:24:12.885018 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.884902 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/612d5222-58c3-4da0-adc4-760f3fcd1843-registry-tls\") pod \"image-registry-6c87577785-px7mf\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " pod="openshift-image-registry/image-registry-6c87577785-px7mf" Sep 08 18:24:12.885018 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.884943 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbkrm\" (UniqueName: \"kubernetes.io/projected/a067ecec-0c84-4299-812c-6441d126104f-kube-api-access-mbkrm\") pod \"ingress-canary-k4dxz\" (UID: \"a067ecec-0c84-4299-812c-6441d126104f\") " pod="openshift-ingress-canary/ingress-canary-k4dxz" Sep 08 18:24:12.885018 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.884987 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/612d5222-58c3-4da0-adc4-760f3fcd1843-image-registry-private-configuration\") pod \"image-registry-6c87577785-px7mf\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " pod="openshift-image-registry/image-registry-6c87577785-px7mf" Sep 08 18:24:12.885018 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.885011 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9de9b23a-f1e9-4b44-87a2-680997e061f0-tmp-dir\") pod \"dns-default-pzzmm\" (UID: \"9de9b23a-f1e9-4b44-87a2-680997e061f0\") " pod="openshift-dns/dns-default-pzzmm" Sep 08 18:24:12.885337 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.885033 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2nc7\" (UniqueName: \"kubernetes.io/projected/9de9b23a-f1e9-4b44-87a2-680997e061f0-kube-api-access-j2nc7\") pod \"dns-default-pzzmm\" (UID: \"9de9b23a-f1e9-4b44-87a2-680997e061f0\") " pod="openshift-dns/dns-default-pzzmm" Sep 08 18:24:12.885337 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.885120 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/612d5222-58c3-4da0-adc4-760f3fcd1843-ca-trust-extracted\") pod \"image-registry-6c87577785-px7mf\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " pod="openshift-image-registry/image-registry-6c87577785-px7mf" Sep 08 18:24:12.885337 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.885256 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9de9b23a-f1e9-4b44-87a2-680997e061f0-config-volume\") pod \"dns-default-pzzmm\" (UID: \"9de9b23a-f1e9-4b44-87a2-680997e061f0\") " pod="openshift-dns/dns-default-pzzmm" Sep 08 18:24:12.885337 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:12.885130 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 18:24:12.885337 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.885288 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/612d5222-58c3-4da0-adc4-760f3fcd1843-registry-certificates\") pod \"image-registry-6c87577785-px7mf\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " pod="openshift-image-registry/image-registry-6c87577785-px7mf" Sep 08 18:24:12.885626 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:12.885355 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9de9b23a-f1e9-4b44-87a2-680997e061f0-metrics-tls podName:9de9b23a-f1e9-4b44-87a2-680997e061f0 nodeName:}" failed. No retries permitted until 2026-09-08 18:24:13.385336232 +0000 UTC m=+35.591321625 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9de9b23a-f1e9-4b44-87a2-680997e061f0-metrics-tls") pod "dns-default-pzzmm" (UID: "9de9b23a-f1e9-4b44-87a2-680997e061f0") : secret "dns-default-metrics-tls" not found Sep 08 18:24:12.885626 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:12.885370 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 08 18:24:12.885626 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:12.885383 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c87577785-px7mf: secret "image-registry-tls" not found Sep 08 18:24:12.885626 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:12.885419 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/612d5222-58c3-4da0-adc4-760f3fcd1843-registry-tls podName:612d5222-58c3-4da0-adc4-760f3fcd1843 nodeName:}" failed. No retries permitted until 2026-09-08 18:24:13.38540738 +0000 UTC m=+35.591392758 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/612d5222-58c3-4da0-adc4-760f3fcd1843-registry-tls") pod "image-registry-6c87577785-px7mf" (UID: "612d5222-58c3-4da0-adc4-760f3fcd1843") : secret "image-registry-tls" not found Sep 08 18:24:12.885829 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.885624 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9de9b23a-f1e9-4b44-87a2-680997e061f0-tmp-dir\") pod \"dns-default-pzzmm\" (UID: \"9de9b23a-f1e9-4b44-87a2-680997e061f0\") " pod="openshift-dns/dns-default-pzzmm" Sep 08 18:24:12.886512 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.886486 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/612d5222-58c3-4da0-adc4-760f3fcd1843-trusted-ca\") pod \"image-registry-6c87577785-px7mf\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " pod="openshift-image-registry/image-registry-6c87577785-px7mf" Sep 08 18:24:12.889661 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.889639 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/612d5222-58c3-4da0-adc4-760f3fcd1843-installation-pull-secrets\") pod \"image-registry-6c87577785-px7mf\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " pod="openshift-image-registry/image-registry-6c87577785-px7mf" Sep 08 18:24:12.889800 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.889658 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/612d5222-58c3-4da0-adc4-760f3fcd1843-image-registry-private-configuration\") pod \"image-registry-6c87577785-px7mf\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " pod="openshift-image-registry/image-registry-6c87577785-px7mf" Sep 08 18:24:12.889800 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.889665 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a067ecec-0c84-4299-812c-6441d126104f-cert\") pod \"ingress-canary-k4dxz\" (UID: \"a067ecec-0c84-4299-812c-6441d126104f\") " pod="openshift-ingress-canary/ingress-canary-k4dxz" Sep 08 18:24:12.893344 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.893316 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2nc7\" (UniqueName: \"kubernetes.io/projected/9de9b23a-f1e9-4b44-87a2-680997e061f0-kube-api-access-j2nc7\") pod \"dns-default-pzzmm\" (UID: \"9de9b23a-f1e9-4b44-87a2-680997e061f0\") " pod="openshift-dns/dns-default-pzzmm" Sep 08 18:24:12.893344 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.893333 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78rp2\" (UniqueName: \"kubernetes.io/projected/612d5222-58c3-4da0-adc4-760f3fcd1843-kube-api-access-78rp2\") pod \"image-registry-6c87577785-px7mf\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " pod="openshift-image-registry/image-registry-6c87577785-px7mf" Sep 08 18:24:12.893344 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.893333 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/612d5222-58c3-4da0-adc4-760f3fcd1843-bound-sa-token\") pod \"image-registry-6c87577785-px7mf\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " pod="openshift-image-registry/image-registry-6c87577785-px7mf" Sep 08 18:24:12.893862 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:12.893842 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbkrm\" (UniqueName: \"kubernetes.io/projected/a067ecec-0c84-4299-812c-6441d126104f-kube-api-access-mbkrm\") pod \"ingress-canary-k4dxz\" (UID: \"a067ecec-0c84-4299-812c-6441d126104f\") " pod="openshift-ingress-canary/ingress-canary-k4dxz" Sep 08 18:24:13.017966 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:13.017917 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-k4dxz" Sep 08 18:24:13.318070 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:13.317975 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qz9vk" Sep 08 18:24:13.318224 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:13.318118 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbksq" Sep 08 18:24:13.318301 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:13.318282 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqthm" Sep 08 18:24:13.320781 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:13.320754 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 08 18:24:13.320781 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:13.320775 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 08 18:24:13.321338 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:13.321318 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jmlkr\"" Sep 08 18:24:13.321612 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:13.321591 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 08 18:24:13.321727 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:13.321637 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 08 18:24:13.321727 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:13.321599 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-frqcg\"" Sep 08 18:24:13.388944 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:13.388909 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9de9b23a-f1e9-4b44-87a2-680997e061f0-metrics-tls\") pod \"dns-default-pzzmm\" (UID: \"9de9b23a-f1e9-4b44-87a2-680997e061f0\") " pod="openshift-dns/dns-default-pzzmm" Sep 08 18:24:13.389139 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:13.389008 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/612d5222-58c3-4da0-adc4-760f3fcd1843-registry-tls\") pod \"image-registry-6c87577785-px7mf\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " pod="openshift-image-registry/image-registry-6c87577785-px7mf" Sep 08 18:24:13.389139 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:13.389098 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 18:24:13.389256 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:13.389171 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 08 18:24:13.389256 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:13.389192 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c87577785-px7mf: secret "image-registry-tls" not found Sep 08 18:24:13.389256 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:13.389175 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9de9b23a-f1e9-4b44-87a2-680997e061f0-metrics-tls podName:9de9b23a-f1e9-4b44-87a2-680997e061f0 nodeName:}" failed. No retries permitted until 2026-09-08 18:24:14.389152493 +0000 UTC m=+36.595137891 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9de9b23a-f1e9-4b44-87a2-680997e061f0-metrics-tls") pod "dns-default-pzzmm" (UID: "9de9b23a-f1e9-4b44-87a2-680997e061f0") : secret "dns-default-metrics-tls" not found Sep 08 18:24:13.389256 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:13.389257 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/612d5222-58c3-4da0-adc4-760f3fcd1843-registry-tls podName:612d5222-58c3-4da0-adc4-760f3fcd1843 nodeName:}" failed. No retries permitted until 2026-09-08 18:24:14.389238934 +0000 UTC m=+36.595224320 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/612d5222-58c3-4da0-adc4-760f3fcd1843-registry-tls") pod "image-registry-6c87577785-px7mf" (UID: "612d5222-58c3-4da0-adc4-760f3fcd1843") : secret "image-registry-tls" not found Sep 08 18:24:13.484814 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:13.484773 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:13.574490 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:13.574401 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:14.359173 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:14.359144 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-k4dxz"] Sep 08 18:24:14.363806 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:24:14.363784 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda067ecec_0c84_4299_812c_6441d126104f.slice/crio-f7da7fb8080b7a6e6ac538732ac35a9e2124e2c409c431c75d78f478b9c8a619 WatchSource:0}: Error finding container f7da7fb8080b7a6e6ac538732ac35a9e2124e2c409c431c75d78f478b9c8a619: Status 404 returned error can't find the container with id f7da7fb8080b7a6e6ac538732ac35a9e2124e2c409c431c75d78f478b9c8a619 Sep 08 18:24:14.397041 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:14.396774 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9de9b23a-f1e9-4b44-87a2-680997e061f0-metrics-tls\") pod \"dns-default-pzzmm\" (UID: \"9de9b23a-f1e9-4b44-87a2-680997e061f0\") " pod="openshift-dns/dns-default-pzzmm" Sep 08 18:24:14.397041 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:14.396850 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/612d5222-58c3-4da0-adc4-760f3fcd1843-registry-tls\") pod \"image-registry-6c87577785-px7mf\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " pod="openshift-image-registry/image-registry-6c87577785-px7mf" Sep 08 18:24:14.397041 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:14.396916 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 18:24:14.397041 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:14.396954 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 08 18:24:14.397041 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:14.396965 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c87577785-px7mf: secret "image-registry-tls" not found Sep 08 18:24:14.397041 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:14.396984 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9de9b23a-f1e9-4b44-87a2-680997e061f0-metrics-tls podName:9de9b23a-f1e9-4b44-87a2-680997e061f0 nodeName:}" failed. No retries permitted until 2026-09-08 18:24:16.396966966 +0000 UTC m=+38.602952364 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9de9b23a-f1e9-4b44-87a2-680997e061f0-metrics-tls") pod "dns-default-pzzmm" (UID: "9de9b23a-f1e9-4b44-87a2-680997e061f0") : secret "dns-default-metrics-tls" not found Sep 08 18:24:14.397041 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:14.397017 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/612d5222-58c3-4da0-adc4-760f3fcd1843-registry-tls podName:612d5222-58c3-4da0-adc4-760f3fcd1843 nodeName:}" failed. No retries permitted until 2026-09-08 18:24:16.39699949 +0000 UTC m=+38.602984871 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/612d5222-58c3-4da0-adc4-760f3fcd1843-registry-tls") pod "image-registry-6c87577785-px7mf" (UID: "612d5222-58c3-4da0-adc4-760f3fcd1843") : secret "image-registry-tls" not found Sep 08 18:24:14.503323 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:14.503141 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-k4dxz" event={"ID":"a067ecec-0c84-4299-812c-6441d126104f","Type":"ContainerStarted","Data":"f7da7fb8080b7a6e6ac538732ac35a9e2124e2c409c431c75d78f478b9c8a619"} Sep 08 18:24:14.506266 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:14.506236 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw886" event={"ID":"a9136afd-27d0-4789-925d-0f7934e8f79b","Type":"ContainerStarted","Data":"df58c06802bca82c774c4b3be0ddb783c0b30b033ca2011050296bf83efd17a1"} Sep 08 18:24:15.512538 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:15.512499 2555 generic.go:356] "Generic (PLEG): container finished" podID="a9136afd-27d0-4789-925d-0f7934e8f79b" containerID="df58c06802bca82c774c4b3be0ddb783c0b30b033ca2011050296bf83efd17a1" exitCode=0 Sep 08 18:24:15.513023 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:15.512545 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw886" event={"ID":"a9136afd-27d0-4789-925d-0f7934e8f79b","Type":"ContainerDied","Data":"df58c06802bca82c774c4b3be0ddb783c0b30b033ca2011050296bf83efd17a1"} Sep 08 18:24:16.412761 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:16.412714 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9de9b23a-f1e9-4b44-87a2-680997e061f0-metrics-tls\") pod \"dns-default-pzzmm\" (UID: \"9de9b23a-f1e9-4b44-87a2-680997e061f0\") " pod="openshift-dns/dns-default-pzzmm" Sep 08 18:24:16.412957 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:16.412807 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/612d5222-58c3-4da0-adc4-760f3fcd1843-registry-tls\") pod \"image-registry-6c87577785-px7mf\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " pod="openshift-image-registry/image-registry-6c87577785-px7mf" Sep 08 18:24:16.412957 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:16.412870 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 18:24:16.412957 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:16.412919 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 08 18:24:16.412957 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:16.412933 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c87577785-px7mf: secret "image-registry-tls" not found Sep 08 18:24:16.412957 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:16.412943 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9de9b23a-f1e9-4b44-87a2-680997e061f0-metrics-tls podName:9de9b23a-f1e9-4b44-87a2-680997e061f0 nodeName:}" failed. No retries permitted until 2026-09-08 18:24:20.412922974 +0000 UTC m=+42.618908369 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9de9b23a-f1e9-4b44-87a2-680997e061f0-metrics-tls") pod "dns-default-pzzmm" (UID: "9de9b23a-f1e9-4b44-87a2-680997e061f0") : secret "dns-default-metrics-tls" not found Sep 08 18:24:16.413219 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:16.412977 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/612d5222-58c3-4da0-adc4-760f3fcd1843-registry-tls podName:612d5222-58c3-4da0-adc4-760f3fcd1843 nodeName:}" failed. No retries permitted until 2026-09-08 18:24:20.412965518 +0000 UTC m=+42.618950894 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/612d5222-58c3-4da0-adc4-760f3fcd1843-registry-tls") pod "image-registry-6c87577785-px7mf" (UID: "612d5222-58c3-4da0-adc4-760f3fcd1843") : secret "image-registry-tls" not found Sep 08 18:24:16.461285 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:16.461210 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:16.518976 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:16.518946 2555 generic.go:356] "Generic (PLEG): container finished" podID="a9136afd-27d0-4789-925d-0f7934e8f79b" containerID="c3bed9ffbe29609aefddcec1a08e7b82eebad714f64b39904832b9ee38ed9fc5" exitCode=0 Sep 08 18:24:16.519565 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:16.518986 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw886" event={"ID":"a9136afd-27d0-4789-925d-0f7934e8f79b","Type":"ContainerDied","Data":"c3bed9ffbe29609aefddcec1a08e7b82eebad714f64b39904832b9ee38ed9fc5"} Sep 08 18:24:16.542373 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:16.542308 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:17.296799 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:17.296775 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:17.397699 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:17.397659 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:17.523600 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:17.523503 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-k4dxz" event={"ID":"a067ecec-0c84-4299-812c-6441d126104f","Type":"ContainerStarted","Data":"afc2f30279370c6dfdea1106653c7be6e971059960c5b4689db020348002836d"} Sep 08 18:24:17.527293 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:17.527270 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vw886" event={"ID":"a9136afd-27d0-4789-925d-0f7934e8f79b","Type":"ContainerStarted","Data":"ac4d74ede86c027bb4e6d0df3ed9f41c30df659bd698b3575b7a8ee26deda0a8"} Sep 08 18:24:17.538264 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:17.538227 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-k4dxz" podStartSLOduration=2.687437754 podStartE2EDuration="5.538213658s" totalImagesPullingTime="2.850775904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:24:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:24:17.53759115 +0000 UTC m=+39.743576545" watchObservedRunningTime="2026-09-08 18:24:17.538213658 +0000 UTC m=+39.744199070" Sep 08 18:24:17.558216 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:17.558175 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vw886" podStartSLOduration=7.164633311 podStartE2EDuration="38.558163565s" totalImagesPullingTime="31.393530254s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 18:23:39 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:24:17.557173876 +0000 UTC m=+39.763159274" watchObservedRunningTime="2026-09-08 18:24:17.558163565 +0000 UTC m=+39.764148963" Sep 08 18:24:17.678485 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:17.678421 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:17.734152 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:17.734120 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:18.682317 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.682279 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8pt6l"] Sep 08 18:24:18.704423 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.704394 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8pt6l"] Sep 08 18:24:18.704579 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.704507 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8pt6l" Sep 08 18:24:18.706565 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.706544 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Sep 08 18:24:18.706804 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.706788 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Sep 08 18:24:18.706887 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.706797 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Sep 08 18:24:18.707134 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.707122 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-k5f42\"" Sep 08 18:24:18.787760 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.787706 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-9n28f"] Sep 08 18:24:18.814183 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.814155 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7bdcf569fd-2mtjd"] Sep 08 18:24:18.814343 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.814293 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-9n28f" Sep 08 18:24:18.816567 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.816542 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Sep 08 18:24:18.816847 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.816832 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-8bpr7\"" Sep 08 18:24:18.816944 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.816929 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Sep 08 18:24:18.817264 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.817251 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Sep 08 18:24:18.817342 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.817255 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Sep 08 18:24:18.825316 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.825291 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Sep 08 18:24:18.829556 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.829536 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c2d77fcf-b01d-4341-8978-41cda35ebe81-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-8pt6l\" (UID: \"c2d77fcf-b01d-4341-8978-41cda35ebe81\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8pt6l" Sep 08 18:24:18.829620 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.829563 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qkr6\" (UniqueName: \"kubernetes.io/projected/c2d77fcf-b01d-4341-8978-41cda35ebe81-kube-api-access-5qkr6\") pod \"cluster-samples-operator-7cdcfcf76c-8pt6l\" (UID: \"c2d77fcf-b01d-4341-8978-41cda35ebe81\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8pt6l" Sep 08 18:24:18.836289 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.836269 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-9n28f"] Sep 08 18:24:18.836380 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.836297 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7bdcf569fd-2mtjd"] Sep 08 18:24:18.836380 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.836365 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7bdcf569fd-2mtjd" Sep 08 18:24:18.838279 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.838252 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Sep 08 18:24:18.838430 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.838360 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Sep 08 18:24:18.838627 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.838434 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 08 18:24:18.838627 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.838494 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-6g2gs\"" Sep 08 18:24:18.838846 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.838829 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Sep 08 18:24:18.838907 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.838851 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 08 18:24:18.838907 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.838831 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Sep 08 18:24:18.889601 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.889567 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-flsvz"] Sep 08 18:24:18.912371 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.912342 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-flsvz"] Sep 08 18:24:18.912512 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.912457 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-flsvz" Sep 08 18:24:18.917280 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.917257 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Sep 08 18:24:18.917443 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.917331 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 08 18:24:18.917558 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.917533 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-sd4xl\"" Sep 08 18:24:18.917676 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.917539 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Sep 08 18:24:18.917858 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.917842 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 08 18:24:18.923832 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.923814 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Sep 08 18:24:18.930519 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.930500 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c44b4985-58c8-4e47-af99-0fd9052f8c7b-default-certificate\") pod \"router-default-7bdcf569fd-2mtjd\" (UID: \"c44b4985-58c8-4e47-af99-0fd9052f8c7b\") " pod="openshift-ingress/router-default-7bdcf569fd-2mtjd" Sep 08 18:24:18.930613 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.930549 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c44b4985-58c8-4e47-af99-0fd9052f8c7b-stats-auth\") pod \"router-default-7bdcf569fd-2mtjd\" (UID: \"c44b4985-58c8-4e47-af99-0fd9052f8c7b\") " pod="openshift-ingress/router-default-7bdcf569fd-2mtjd" Sep 08 18:24:18.930613 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.930570 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/911c61b8-766b-4ae8-b046-784e22e85889-config\") pod \"console-operator-85b454874b-9n28f\" (UID: \"911c61b8-766b-4ae8-b046-784e22e85889\") " pod="openshift-console-operator/console-operator-85b454874b-9n28f" Sep 08 18:24:18.930613 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.930585 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxldq\" (UniqueName: \"kubernetes.io/projected/c44b4985-58c8-4e47-af99-0fd9052f8c7b-kube-api-access-pxldq\") pod \"router-default-7bdcf569fd-2mtjd\" (UID: \"c44b4985-58c8-4e47-af99-0fd9052f8c7b\") " pod="openshift-ingress/router-default-7bdcf569fd-2mtjd" Sep 08 18:24:18.930811 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.930735 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/911c61b8-766b-4ae8-b046-784e22e85889-trusted-ca\") pod \"console-operator-85b454874b-9n28f\" (UID: \"911c61b8-766b-4ae8-b046-784e22e85889\") " pod="openshift-console-operator/console-operator-85b454874b-9n28f" Sep 08 18:24:18.930811 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.930765 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqkpr\" (UniqueName: \"kubernetes.io/projected/911c61b8-766b-4ae8-b046-784e22e85889-kube-api-access-vqkpr\") pod \"console-operator-85b454874b-9n28f\" (UID: \"911c61b8-766b-4ae8-b046-784e22e85889\") " pod="openshift-console-operator/console-operator-85b454874b-9n28f" Sep 08 18:24:18.930886 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.930820 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c2d77fcf-b01d-4341-8978-41cda35ebe81-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-8pt6l\" (UID: \"c2d77fcf-b01d-4341-8978-41cda35ebe81\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8pt6l" Sep 08 18:24:18.930886 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.930837 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qkr6\" (UniqueName: \"kubernetes.io/projected/c2d77fcf-b01d-4341-8978-41cda35ebe81-kube-api-access-5qkr6\") pod \"cluster-samples-operator-7cdcfcf76c-8pt6l\" (UID: \"c2d77fcf-b01d-4341-8978-41cda35ebe81\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8pt6l" Sep 08 18:24:18.930886 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.930850 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/911c61b8-766b-4ae8-b046-784e22e85889-serving-cert\") pod \"console-operator-85b454874b-9n28f\" (UID: \"911c61b8-766b-4ae8-b046-784e22e85889\") " pod="openshift-console-operator/console-operator-85b454874b-9n28f" Sep 08 18:24:18.931007 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:18.930925 2555 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 08 18:24:18.931007 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.930956 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c44b4985-58c8-4e47-af99-0fd9052f8c7b-service-ca-bundle\") pod \"router-default-7bdcf569fd-2mtjd\" (UID: \"c44b4985-58c8-4e47-af99-0fd9052f8c7b\") " pod="openshift-ingress/router-default-7bdcf569fd-2mtjd" Sep 08 18:24:18.931007 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:18.930987 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2d77fcf-b01d-4341-8978-41cda35ebe81-samples-operator-tls podName:c2d77fcf-b01d-4341-8978-41cda35ebe81 nodeName:}" failed. No retries permitted until 2026-09-08 18:24:19.430972761 +0000 UTC m=+41.636958137 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c2d77fcf-b01d-4341-8978-41cda35ebe81-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-8pt6l" (UID: "c2d77fcf-b01d-4341-8978-41cda35ebe81") : secret "samples-operator-tls" not found Sep 08 18:24:18.931237 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.931005 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c44b4985-58c8-4e47-af99-0fd9052f8c7b-metrics-certs\") pod \"router-default-7bdcf569fd-2mtjd\" (UID: \"c44b4985-58c8-4e47-af99-0fd9052f8c7b\") " pod="openshift-ingress/router-default-7bdcf569fd-2mtjd" Sep 08 18:24:18.948442 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:18.948373 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qkr6\" (UniqueName: \"kubernetes.io/projected/c2d77fcf-b01d-4341-8978-41cda35ebe81-kube-api-access-5qkr6\") pod \"cluster-samples-operator-7cdcfcf76c-8pt6l\" (UID: \"c2d77fcf-b01d-4341-8978-41cda35ebe81\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8pt6l" Sep 08 18:24:19.002066 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.002028 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:19.031542 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.031505 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41932272-e547-4229-be69-f99a4096637e-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-flsvz\" (UID: \"41932272-e547-4229-be69-f99a4096637e\") " pod="openshift-insights/insights-operator-76bcd6ccf5-flsvz" Sep 08 18:24:19.031717 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.031558 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqkpr\" (UniqueName: \"kubernetes.io/projected/911c61b8-766b-4ae8-b046-784e22e85889-kube-api-access-vqkpr\") pod \"console-operator-85b454874b-9n28f\" (UID: \"911c61b8-766b-4ae8-b046-784e22e85889\") " pod="openshift-console-operator/console-operator-85b454874b-9n28f" Sep 08 18:24:19.031717 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.031588 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5s2tl\" (UniqueName: \"kubernetes.io/projected/41932272-e547-4229-be69-f99a4096637e-kube-api-access-5s2tl\") pod \"insights-operator-76bcd6ccf5-flsvz\" (UID: \"41932272-e547-4229-be69-f99a4096637e\") " pod="openshift-insights/insights-operator-76bcd6ccf5-flsvz" Sep 08 18:24:19.031717 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.031622 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c44b4985-58c8-4e47-af99-0fd9052f8c7b-service-ca-bundle\") pod \"router-default-7bdcf569fd-2mtjd\" (UID: \"c44b4985-58c8-4e47-af99-0fd9052f8c7b\") " pod="openshift-ingress/router-default-7bdcf569fd-2mtjd" Sep 08 18:24:19.031717 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.031641 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c44b4985-58c8-4e47-af99-0fd9052f8c7b-metrics-certs\") pod \"router-default-7bdcf569fd-2mtjd\" (UID: \"c44b4985-58c8-4e47-af99-0fd9052f8c7b\") " pod="openshift-ingress/router-default-7bdcf569fd-2mtjd" Sep 08 18:24:19.031717 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.031658 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/41932272-e547-4229-be69-f99a4096637e-snapshots\") pod \"insights-operator-76bcd6ccf5-flsvz\" (UID: \"41932272-e547-4229-be69-f99a4096637e\") " pod="openshift-insights/insights-operator-76bcd6ccf5-flsvz" Sep 08 18:24:19.031942 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.031738 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c44b4985-58c8-4e47-af99-0fd9052f8c7b-default-certificate\") pod \"router-default-7bdcf569fd-2mtjd\" (UID: \"c44b4985-58c8-4e47-af99-0fd9052f8c7b\") " pod="openshift-ingress/router-default-7bdcf569fd-2mtjd" Sep 08 18:24:19.031942 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:19.031763 2555 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 08 18:24:19.031942 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.031772 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41932272-e547-4229-be69-f99a4096637e-tmp\") pod \"insights-operator-76bcd6ccf5-flsvz\" (UID: \"41932272-e547-4229-be69-f99a4096637e\") " pod="openshift-insights/insights-operator-76bcd6ccf5-flsvz" Sep 08 18:24:19.031942 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:19.031816 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c44b4985-58c8-4e47-af99-0fd9052f8c7b-service-ca-bundle podName:c44b4985-58c8-4e47-af99-0fd9052f8c7b nodeName:}" failed. No retries permitted until 2026-09-08 18:24:19.53179589 +0000 UTC m=+41.737781288 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c44b4985-58c8-4e47-af99-0fd9052f8c7b-service-ca-bundle") pod "router-default-7bdcf569fd-2mtjd" (UID: "c44b4985-58c8-4e47-af99-0fd9052f8c7b") : configmap references non-existent config key: service-ca.crt Sep 08 18:24:19.031942 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.031839 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/911c61b8-766b-4ae8-b046-784e22e85889-config\") pod \"console-operator-85b454874b-9n28f\" (UID: \"911c61b8-766b-4ae8-b046-784e22e85889\") " pod="openshift-console-operator/console-operator-85b454874b-9n28f" Sep 08 18:24:19.031942 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:19.031860 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c44b4985-58c8-4e47-af99-0fd9052f8c7b-metrics-certs podName:c44b4985-58c8-4e47-af99-0fd9052f8c7b nodeName:}" failed. No retries permitted until 2026-09-08 18:24:19.531850518 +0000 UTC m=+41.737835900 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c44b4985-58c8-4e47-af99-0fd9052f8c7b-metrics-certs") pod "router-default-7bdcf569fd-2mtjd" (UID: "c44b4985-58c8-4e47-af99-0fd9052f8c7b") : secret "router-metrics-certs-default" not found Sep 08 18:24:19.031942 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.031923 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxldq\" (UniqueName: \"kubernetes.io/projected/c44b4985-58c8-4e47-af99-0fd9052f8c7b-kube-api-access-pxldq\") pod \"router-default-7bdcf569fd-2mtjd\" (UID: \"c44b4985-58c8-4e47-af99-0fd9052f8c7b\") " pod="openshift-ingress/router-default-7bdcf569fd-2mtjd" Sep 08 18:24:19.032265 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.031988 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/911c61b8-766b-4ae8-b046-784e22e85889-trusted-ca\") pod \"console-operator-85b454874b-9n28f\" (UID: \"911c61b8-766b-4ae8-b046-784e22e85889\") " pod="openshift-console-operator/console-operator-85b454874b-9n28f" Sep 08 18:24:19.032265 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.032024 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/41932272-e547-4229-be69-f99a4096637e-serving-cert\") pod \"insights-operator-76bcd6ccf5-flsvz\" (UID: \"41932272-e547-4229-be69-f99a4096637e\") " pod="openshift-insights/insights-operator-76bcd6ccf5-flsvz" Sep 08 18:24:19.032265 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.032053 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/911c61b8-766b-4ae8-b046-784e22e85889-serving-cert\") pod \"console-operator-85b454874b-9n28f\" (UID: \"911c61b8-766b-4ae8-b046-784e22e85889\") " pod="openshift-console-operator/console-operator-85b454874b-9n28f" Sep 08 18:24:19.032265 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.032118 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c44b4985-58c8-4e47-af99-0fd9052f8c7b-stats-auth\") pod \"router-default-7bdcf569fd-2mtjd\" (UID: \"c44b4985-58c8-4e47-af99-0fd9052f8c7b\") " pod="openshift-ingress/router-default-7bdcf569fd-2mtjd" Sep 08 18:24:19.032265 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.032143 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41932272-e547-4229-be69-f99a4096637e-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-flsvz\" (UID: \"41932272-e547-4229-be69-f99a4096637e\") " pod="openshift-insights/insights-operator-76bcd6ccf5-flsvz" Sep 08 18:24:19.032673 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.032644 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/911c61b8-766b-4ae8-b046-784e22e85889-config\") pod \"console-operator-85b454874b-9n28f\" (UID: \"911c61b8-766b-4ae8-b046-784e22e85889\") " pod="openshift-console-operator/console-operator-85b454874b-9n28f" Sep 08 18:24:19.032921 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.032901 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/911c61b8-766b-4ae8-b046-784e22e85889-trusted-ca\") pod \"console-operator-85b454874b-9n28f\" (UID: \"911c61b8-766b-4ae8-b046-784e22e85889\") " pod="openshift-console-operator/console-operator-85b454874b-9n28f" Sep 08 18:24:19.034603 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.034575 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/911c61b8-766b-4ae8-b046-784e22e85889-serving-cert\") pod \"console-operator-85b454874b-9n28f\" (UID: \"911c61b8-766b-4ae8-b046-784e22e85889\") " pod="openshift-console-operator/console-operator-85b454874b-9n28f" Sep 08 18:24:19.034773 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.034755 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c44b4985-58c8-4e47-af99-0fd9052f8c7b-stats-auth\") pod \"router-default-7bdcf569fd-2mtjd\" (UID: \"c44b4985-58c8-4e47-af99-0fd9052f8c7b\") " pod="openshift-ingress/router-default-7bdcf569fd-2mtjd" Sep 08 18:24:19.034858 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.034840 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c44b4985-58c8-4e47-af99-0fd9052f8c7b-default-certificate\") pod \"router-default-7bdcf569fd-2mtjd\" (UID: \"c44b4985-58c8-4e47-af99-0fd9052f8c7b\") " pod="openshift-ingress/router-default-7bdcf569fd-2mtjd" Sep 08 18:24:19.039967 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.039945 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqkpr\" (UniqueName: \"kubernetes.io/projected/911c61b8-766b-4ae8-b046-784e22e85889-kube-api-access-vqkpr\") pod \"console-operator-85b454874b-9n28f\" (UID: \"911c61b8-766b-4ae8-b046-784e22e85889\") " pod="openshift-console-operator/console-operator-85b454874b-9n28f" Sep 08 18:24:19.040078 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.040063 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxldq\" (UniqueName: \"kubernetes.io/projected/c44b4985-58c8-4e47-af99-0fd9052f8c7b-kube-api-access-pxldq\") pod \"router-default-7bdcf569fd-2mtjd\" (UID: \"c44b4985-58c8-4e47-af99-0fd9052f8c7b\") " pod="openshift-ingress/router-default-7bdcf569fd-2mtjd" Sep 08 18:24:19.070392 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.070361 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:19.124396 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.124360 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-9n28f" Sep 08 18:24:19.132667 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.132642 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5s2tl\" (UniqueName: \"kubernetes.io/projected/41932272-e547-4229-be69-f99a4096637e-kube-api-access-5s2tl\") pod \"insights-operator-76bcd6ccf5-flsvz\" (UID: \"41932272-e547-4229-be69-f99a4096637e\") " pod="openshift-insights/insights-operator-76bcd6ccf5-flsvz" Sep 08 18:24:19.132754 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.132725 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/41932272-e547-4229-be69-f99a4096637e-snapshots\") pod \"insights-operator-76bcd6ccf5-flsvz\" (UID: \"41932272-e547-4229-be69-f99a4096637e\") " pod="openshift-insights/insights-operator-76bcd6ccf5-flsvz" Sep 08 18:24:19.132754 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.132750 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41932272-e547-4229-be69-f99a4096637e-tmp\") pod \"insights-operator-76bcd6ccf5-flsvz\" (UID: \"41932272-e547-4229-be69-f99a4096637e\") " pod="openshift-insights/insights-operator-76bcd6ccf5-flsvz" Sep 08 18:24:19.132905 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.132784 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/41932272-e547-4229-be69-f99a4096637e-serving-cert\") pod \"insights-operator-76bcd6ccf5-flsvz\" (UID: \"41932272-e547-4229-be69-f99a4096637e\") " pod="openshift-insights/insights-operator-76bcd6ccf5-flsvz" Sep 08 18:24:19.132905 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.132835 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41932272-e547-4229-be69-f99a4096637e-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-flsvz\" (UID: \"41932272-e547-4229-be69-f99a4096637e\") " pod="openshift-insights/insights-operator-76bcd6ccf5-flsvz" Sep 08 18:24:19.132905 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.132868 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41932272-e547-4229-be69-f99a4096637e-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-flsvz\" (UID: \"41932272-e547-4229-be69-f99a4096637e\") " pod="openshift-insights/insights-operator-76bcd6ccf5-flsvz" Sep 08 18:24:19.133379 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.133350 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41932272-e547-4229-be69-f99a4096637e-tmp\") pod \"insights-operator-76bcd6ccf5-flsvz\" (UID: \"41932272-e547-4229-be69-f99a4096637e\") " pod="openshift-insights/insights-operator-76bcd6ccf5-flsvz" Sep 08 18:24:19.133379 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.133363 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/41932272-e547-4229-be69-f99a4096637e-snapshots\") pod \"insights-operator-76bcd6ccf5-flsvz\" (UID: \"41932272-e547-4229-be69-f99a4096637e\") " pod="openshift-insights/insights-operator-76bcd6ccf5-flsvz" Sep 08 18:24:19.133495 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.133420 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41932272-e547-4229-be69-f99a4096637e-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-flsvz\" (UID: \"41932272-e547-4229-be69-f99a4096637e\") " pod="openshift-insights/insights-operator-76bcd6ccf5-flsvz" Sep 08 18:24:19.133676 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.133659 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41932272-e547-4229-be69-f99a4096637e-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-flsvz\" (UID: \"41932272-e547-4229-be69-f99a4096637e\") " pod="openshift-insights/insights-operator-76bcd6ccf5-flsvz" Sep 08 18:24:19.135194 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.135176 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/41932272-e547-4229-be69-f99a4096637e-serving-cert\") pod \"insights-operator-76bcd6ccf5-flsvz\" (UID: \"41932272-e547-4229-be69-f99a4096637e\") " pod="openshift-insights/insights-operator-76bcd6ccf5-flsvz" Sep 08 18:24:19.140754 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.140736 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5s2tl\" (UniqueName: \"kubernetes.io/projected/41932272-e547-4229-be69-f99a4096637e-kube-api-access-5s2tl\") pod \"insights-operator-76bcd6ccf5-flsvz\" (UID: \"41932272-e547-4229-be69-f99a4096637e\") " pod="openshift-insights/insights-operator-76bcd6ccf5-flsvz" Sep 08 18:24:19.170855 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.170666 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:19.221200 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.221158 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-flsvz" Sep 08 18:24:19.226884 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.226844 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:19.427074 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.427042 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-9n28f"] Sep 08 18:24:19.430021 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:24:19.429983 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod911c61b8_766b_4ae8_b046_784e22e85889.slice/crio-9926c967f4ec85846a5afa2a57afdd611f0566fb2b276be08c2c07240f992d0a WatchSource:0}: Error finding container 9926c967f4ec85846a5afa2a57afdd611f0566fb2b276be08c2c07240f992d0a: Status 404 returned error can't find the container with id 9926c967f4ec85846a5afa2a57afdd611f0566fb2b276be08c2c07240f992d0a Sep 08 18:24:19.435678 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.435658 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c2d77fcf-b01d-4341-8978-41cda35ebe81-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-8pt6l\" (UID: \"c2d77fcf-b01d-4341-8978-41cda35ebe81\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8pt6l" Sep 08 18:24:19.435791 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:19.435776 2555 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 08 18:24:19.435838 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:19.435834 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2d77fcf-b01d-4341-8978-41cda35ebe81-samples-operator-tls podName:c2d77fcf-b01d-4341-8978-41cda35ebe81 nodeName:}" failed. No retries permitted until 2026-09-08 18:24:20.435817424 +0000 UTC m=+42.641802802 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c2d77fcf-b01d-4341-8978-41cda35ebe81-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-8pt6l" (UID: "c2d77fcf-b01d-4341-8978-41cda35ebe81") : secret "samples-operator-tls" not found Sep 08 18:24:19.533061 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.532975 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-9n28f" event={"ID":"911c61b8-766b-4ae8-b046-784e22e85889","Type":"ContainerStarted","Data":"9926c967f4ec85846a5afa2a57afdd611f0566fb2b276be08c2c07240f992d0a"} Sep 08 18:24:19.536655 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.536625 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-flsvz"] Sep 08 18:24:19.536823 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.536710 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c44b4985-58c8-4e47-af99-0fd9052f8c7b-service-ca-bundle\") pod \"router-default-7bdcf569fd-2mtjd\" (UID: \"c44b4985-58c8-4e47-af99-0fd9052f8c7b\") " pod="openshift-ingress/router-default-7bdcf569fd-2mtjd" Sep 08 18:24:19.536823 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:19.536738 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c44b4985-58c8-4e47-af99-0fd9052f8c7b-metrics-certs\") pod \"router-default-7bdcf569fd-2mtjd\" (UID: \"c44b4985-58c8-4e47-af99-0fd9052f8c7b\") " pod="openshift-ingress/router-default-7bdcf569fd-2mtjd" Sep 08 18:24:19.536932 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:19.536871 2555 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 08 18:24:19.536932 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:19.536881 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c44b4985-58c8-4e47-af99-0fd9052f8c7b-service-ca-bundle podName:c44b4985-58c8-4e47-af99-0fd9052f8c7b nodeName:}" failed. No retries permitted until 2026-09-08 18:24:20.536857272 +0000 UTC m=+42.742842669 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c44b4985-58c8-4e47-af99-0fd9052f8c7b-service-ca-bundle") pod "router-default-7bdcf569fd-2mtjd" (UID: "c44b4985-58c8-4e47-af99-0fd9052f8c7b") : configmap references non-existent config key: service-ca.crt Sep 08 18:24:19.536932 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:19.536917 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c44b4985-58c8-4e47-af99-0fd9052f8c7b-metrics-certs podName:c44b4985-58c8-4e47-af99-0fd9052f8c7b nodeName:}" failed. No retries permitted until 2026-09-08 18:24:20.536905986 +0000 UTC m=+42.742891399 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c44b4985-58c8-4e47-af99-0fd9052f8c7b-metrics-certs") pod "router-default-7bdcf569fd-2mtjd" (UID: "c44b4985-58c8-4e47-af99-0fd9052f8c7b") : secret "router-metrics-certs-default" not found Sep 08 18:24:19.540726 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:24:19.540671 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41932272_e547_4229_be69_f99a4096637e.slice/crio-5355f1f14aa8ba46f71057851693ff671fc945ada26ed0d0b21c8e9318053cf5 WatchSource:0}: Error finding container 5355f1f14aa8ba46f71057851693ff671fc945ada26ed0d0b21c8e9318053cf5: Status 404 returned error can't find the container with id 5355f1f14aa8ba46f71057851693ff671fc945ada26ed0d0b21c8e9318053cf5 Sep 08 18:24:20.447973 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:20.447933 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9de9b23a-f1e9-4b44-87a2-680997e061f0-metrics-tls\") pod \"dns-default-pzzmm\" (UID: \"9de9b23a-f1e9-4b44-87a2-680997e061f0\") " pod="openshift-dns/dns-default-pzzmm" Sep 08 18:24:20.448474 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:20.448007 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/612d5222-58c3-4da0-adc4-760f3fcd1843-registry-tls\") pod \"image-registry-6c87577785-px7mf\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " pod="openshift-image-registry/image-registry-6c87577785-px7mf" Sep 08 18:24:20.448474 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:20.448074 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c2d77fcf-b01d-4341-8978-41cda35ebe81-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-8pt6l\" (UID: \"c2d77fcf-b01d-4341-8978-41cda35ebe81\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8pt6l" Sep 08 18:24:20.448474 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:20.448119 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 18:24:20.448474 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:20.448187 2555 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 08 18:24:20.448474 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:20.448186 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 08 18:24:20.448474 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:20.448209 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9de9b23a-f1e9-4b44-87a2-680997e061f0-metrics-tls podName:9de9b23a-f1e9-4b44-87a2-680997e061f0 nodeName:}" failed. No retries permitted until 2026-09-08 18:24:28.448192709 +0000 UTC m=+50.654178085 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9de9b23a-f1e9-4b44-87a2-680997e061f0-metrics-tls") pod "dns-default-pzzmm" (UID: "9de9b23a-f1e9-4b44-87a2-680997e061f0") : secret "dns-default-metrics-tls" not found Sep 08 18:24:20.448474 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:20.448212 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c87577785-px7mf: secret "image-registry-tls" not found Sep 08 18:24:20.448474 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:20.448254 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2d77fcf-b01d-4341-8978-41cda35ebe81-samples-operator-tls podName:c2d77fcf-b01d-4341-8978-41cda35ebe81 nodeName:}" failed. No retries permitted until 2026-09-08 18:24:22.448237542 +0000 UTC m=+44.654222922 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c2d77fcf-b01d-4341-8978-41cda35ebe81-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-8pt6l" (UID: "c2d77fcf-b01d-4341-8978-41cda35ebe81") : secret "samples-operator-tls" not found Sep 08 18:24:20.448474 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:20.448276 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/612d5222-58c3-4da0-adc4-760f3fcd1843-registry-tls podName:612d5222-58c3-4da0-adc4-760f3fcd1843 nodeName:}" failed. No retries permitted until 2026-09-08 18:24:28.448260485 +0000 UTC m=+50.654245863 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/612d5222-58c3-4da0-adc4-760f3fcd1843-registry-tls") pod "image-registry-6c87577785-px7mf" (UID: "612d5222-58c3-4da0-adc4-760f3fcd1843") : secret "image-registry-tls" not found Sep 08 18:24:20.539100 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:20.539059 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-flsvz" event={"ID":"41932272-e547-4229-be69-f99a4096637e","Type":"ContainerStarted","Data":"5355f1f14aa8ba46f71057851693ff671fc945ada26ed0d0b21c8e9318053cf5"} Sep 08 18:24:20.548767 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:20.548729 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c44b4985-58c8-4e47-af99-0fd9052f8c7b-service-ca-bundle\") pod \"router-default-7bdcf569fd-2mtjd\" (UID: \"c44b4985-58c8-4e47-af99-0fd9052f8c7b\") " pod="openshift-ingress/router-default-7bdcf569fd-2mtjd" Sep 08 18:24:20.548767 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:20.548773 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c44b4985-58c8-4e47-af99-0fd9052f8c7b-metrics-certs\") pod \"router-default-7bdcf569fd-2mtjd\" (UID: \"c44b4985-58c8-4e47-af99-0fd9052f8c7b\") " pod="openshift-ingress/router-default-7bdcf569fd-2mtjd" Sep 08 18:24:20.549004 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:20.548893 2555 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 08 18:24:20.549004 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:20.548929 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c44b4985-58c8-4e47-af99-0fd9052f8c7b-service-ca-bundle podName:c44b4985-58c8-4e47-af99-0fd9052f8c7b nodeName:}" failed. No retries permitted until 2026-09-08 18:24:22.548906324 +0000 UTC m=+44.754891714 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c44b4985-58c8-4e47-af99-0fd9052f8c7b-service-ca-bundle") pod "router-default-7bdcf569fd-2mtjd" (UID: "c44b4985-58c8-4e47-af99-0fd9052f8c7b") : configmap references non-existent config key: service-ca.crt Sep 08 18:24:20.549004 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:20.548958 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c44b4985-58c8-4e47-af99-0fd9052f8c7b-metrics-certs podName:c44b4985-58c8-4e47-af99-0fd9052f8c7b nodeName:}" failed. No retries permitted until 2026-09-08 18:24:22.548947962 +0000 UTC m=+44.754933345 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c44b4985-58c8-4e47-af99-0fd9052f8c7b-metrics-certs") pod "router-default-7bdcf569fd-2mtjd" (UID: "c44b4985-58c8-4e47-af99-0fd9052f8c7b") : secret "router-metrics-certs-default" not found Sep 08 18:24:22.468204 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:22.468162 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c2d77fcf-b01d-4341-8978-41cda35ebe81-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-8pt6l\" (UID: \"c2d77fcf-b01d-4341-8978-41cda35ebe81\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8pt6l" Sep 08 18:24:22.468657 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:22.468366 2555 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 08 18:24:22.468657 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:22.468467 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2d77fcf-b01d-4341-8978-41cda35ebe81-samples-operator-tls podName:c2d77fcf-b01d-4341-8978-41cda35ebe81 nodeName:}" failed. No retries permitted until 2026-09-08 18:24:26.468445363 +0000 UTC m=+48.674430743 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c2d77fcf-b01d-4341-8978-41cda35ebe81-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-8pt6l" (UID: "c2d77fcf-b01d-4341-8978-41cda35ebe81") : secret "samples-operator-tls" not found Sep 08 18:24:22.569049 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:22.569014 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c44b4985-58c8-4e47-af99-0fd9052f8c7b-service-ca-bundle\") pod \"router-default-7bdcf569fd-2mtjd\" (UID: \"c44b4985-58c8-4e47-af99-0fd9052f8c7b\") " pod="openshift-ingress/router-default-7bdcf569fd-2mtjd" Sep 08 18:24:22.569049 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:22.569049 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c44b4985-58c8-4e47-af99-0fd9052f8c7b-metrics-certs\") pod \"router-default-7bdcf569fd-2mtjd\" (UID: \"c44b4985-58c8-4e47-af99-0fd9052f8c7b\") " pod="openshift-ingress/router-default-7bdcf569fd-2mtjd" Sep 08 18:24:22.569236 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:22.569158 2555 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 08 18:24:22.569236 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:22.569178 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c44b4985-58c8-4e47-af99-0fd9052f8c7b-service-ca-bundle podName:c44b4985-58c8-4e47-af99-0fd9052f8c7b nodeName:}" failed. No retries permitted until 2026-09-08 18:24:26.569161933 +0000 UTC m=+48.775147308 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c44b4985-58c8-4e47-af99-0fd9052f8c7b-service-ca-bundle") pod "router-default-7bdcf569fd-2mtjd" (UID: "c44b4985-58c8-4e47-af99-0fd9052f8c7b") : configmap references non-existent config key: service-ca.crt Sep 08 18:24:22.569236 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:22.569201 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c44b4985-58c8-4e47-af99-0fd9052f8c7b-metrics-certs podName:c44b4985-58c8-4e47-af99-0fd9052f8c7b nodeName:}" failed. No retries permitted until 2026-09-08 18:24:26.569194558 +0000 UTC m=+48.775179934 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c44b4985-58c8-4e47-af99-0fd9052f8c7b-metrics-certs") pod "router-default-7bdcf569fd-2mtjd" (UID: "c44b4985-58c8-4e47-af99-0fd9052f8c7b") : secret "router-metrics-certs-default" not found Sep 08 18:24:23.520334 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:23.520157 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:23.549264 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:23.549238 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9n28f_911c61b8-766b-4ae8-b046-784e22e85889/console-operator/0.log" Sep 08 18:24:23.549413 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:23.549274 2555 generic.go:356] "Generic (PLEG): container finished" podID="911c61b8-766b-4ae8-b046-784e22e85889" containerID="cefc46a5d82b284f217c5c0e9631004040758cfe2ab989e1eecafef97c12f8c9" exitCode=255 Sep 08 18:24:23.549413 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:23.549308 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-9n28f" event={"ID":"911c61b8-766b-4ae8-b046-784e22e85889","Type":"ContainerDied","Data":"cefc46a5d82b284f217c5c0e9631004040758cfe2ab989e1eecafef97c12f8c9"} Sep 08 18:24:23.549585 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:23.549567 2555 scope.go:117] "RemoveContainer" containerID="cefc46a5d82b284f217c5c0e9631004040758cfe2ab989e1eecafef97c12f8c9" Sep 08 18:24:23.551124 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:23.551077 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-flsvz" event={"ID":"41932272-e547-4229-be69-f99a4096637e","Type":"ContainerStarted","Data":"25c4e13bf7bd9e068a49832fe2515f595a75cf42202eaa69de710f9908323f5c"} Sep 08 18:24:23.578904 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:23.578857 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-flsvz" podStartSLOduration=2.175466547 podStartE2EDuration="5.578839341s" totalImagesPullingTime="3.403372794s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:24:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:24:23.577951208 +0000 UTC m=+45.783936606" watchObservedRunningTime="2026-09-08 18:24:23.578839341 +0000 UTC m=+45.784824738" Sep 08 18:24:23.583012 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:23.582972 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:24.301727 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:24.301658 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-rhzfq"] Sep 08 18:24:24.310751 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:24.310721 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rhzfq" Sep 08 18:24:24.312349 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:24.312315 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:24.312984 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:24.312964 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-srf5g\"" Sep 08 18:24:24.312984 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:24.312981 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 08 18:24:24.313420 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:24.313405 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 08 18:24:24.315835 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:24.315813 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rhzfq"] Sep 08 18:24:24.402641 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:24.402604 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:24.475543 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:24.475506 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:24.486736 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:24.486699 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/29df756b-120f-4026-ac1f-b14507717d42-crio-socket\") pod \"insights-runtime-extractor-rhzfq\" (UID: \"29df756b-120f-4026-ac1f-b14507717d42\") " pod="openshift-insights/insights-runtime-extractor-rhzfq" Sep 08 18:24:24.486851 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:24.486746 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/29df756b-120f-4026-ac1f-b14507717d42-data-volume\") pod \"insights-runtime-extractor-rhzfq\" (UID: \"29df756b-120f-4026-ac1f-b14507717d42\") " pod="openshift-insights/insights-runtime-extractor-rhzfq" Sep 08 18:24:24.486851 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:24.486837 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29df756b-120f-4026-ac1f-b14507717d42-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rhzfq\" (UID: \"29df756b-120f-4026-ac1f-b14507717d42\") " pod="openshift-insights/insights-runtime-extractor-rhzfq" Sep 08 18:24:24.486938 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:24.486887 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khbsc\" (UniqueName: \"kubernetes.io/projected/29df756b-120f-4026-ac1f-b14507717d42-kube-api-access-khbsc\") pod \"insights-runtime-extractor-rhzfq\" (UID: \"29df756b-120f-4026-ac1f-b14507717d42\") " pod="openshift-insights/insights-runtime-extractor-rhzfq" Sep 08 18:24:24.486938 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:24.486912 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/29df756b-120f-4026-ac1f-b14507717d42-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rhzfq\" (UID: \"29df756b-120f-4026-ac1f-b14507717d42\") " pod="openshift-insights/insights-runtime-extractor-rhzfq" Sep 08 18:24:24.532247 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:24.532213 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:24.554947 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:24.554873 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9n28f_911c61b8-766b-4ae8-b046-784e22e85889/console-operator/1.log" Sep 08 18:24:24.555334 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:24.555316 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9n28f_911c61b8-766b-4ae8-b046-784e22e85889/console-operator/0.log" Sep 08 18:24:24.555435 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:24.555358 2555 generic.go:356] "Generic (PLEG): container finished" podID="911c61b8-766b-4ae8-b046-784e22e85889" containerID="601b310cdcd96939941a23b18faf640731aac9e5dca5a1b478cf85f83d9c1d2e" exitCode=255 Sep 08 18:24:24.555496 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:24.555431 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-9n28f" event={"ID":"911c61b8-766b-4ae8-b046-784e22e85889","Type":"ContainerDied","Data":"601b310cdcd96939941a23b18faf640731aac9e5dca5a1b478cf85f83d9c1d2e"} Sep 08 18:24:24.555545 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:24.555494 2555 scope.go:117] "RemoveContainer" containerID="cefc46a5d82b284f217c5c0e9631004040758cfe2ab989e1eecafef97c12f8c9" Sep 08 18:24:24.555726 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:24.555708 2555 scope.go:117] "RemoveContainer" containerID="601b310cdcd96939941a23b18faf640731aac9e5dca5a1b478cf85f83d9c1d2e" Sep 08 18:24:24.555914 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:24.555889 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-9n28f_openshift-console-operator(911c61b8-766b-4ae8-b046-784e22e85889)\"" pod="openshift-console-operator/console-operator-85b454874b-9n28f" podUID="911c61b8-766b-4ae8-b046-784e22e85889" Sep 08 18:24:24.587496 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:24.587461 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/29df756b-120f-4026-ac1f-b14507717d42-crio-socket\") pod \"insights-runtime-extractor-rhzfq\" (UID: \"29df756b-120f-4026-ac1f-b14507717d42\") " pod="openshift-insights/insights-runtime-extractor-rhzfq" Sep 08 18:24:24.587629 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:24.587505 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/29df756b-120f-4026-ac1f-b14507717d42-data-volume\") pod \"insights-runtime-extractor-rhzfq\" (UID: \"29df756b-120f-4026-ac1f-b14507717d42\") " pod="openshift-insights/insights-runtime-extractor-rhzfq" Sep 08 18:24:24.587629 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:24.587559 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29df756b-120f-4026-ac1f-b14507717d42-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rhzfq\" (UID: \"29df756b-120f-4026-ac1f-b14507717d42\") " pod="openshift-insights/insights-runtime-extractor-rhzfq" Sep 08 18:24:24.587629 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:24.587582 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khbsc\" (UniqueName: \"kubernetes.io/projected/29df756b-120f-4026-ac1f-b14507717d42-kube-api-access-khbsc\") pod \"insights-runtime-extractor-rhzfq\" (UID: \"29df756b-120f-4026-ac1f-b14507717d42\") " pod="openshift-insights/insights-runtime-extractor-rhzfq" Sep 08 18:24:24.587629 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:24.587601 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/29df756b-120f-4026-ac1f-b14507717d42-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rhzfq\" (UID: \"29df756b-120f-4026-ac1f-b14507717d42\") " pod="openshift-insights/insights-runtime-extractor-rhzfq" Sep 08 18:24:24.587804 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:24.587714 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 08 18:24:24.587804 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:24.587712 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/29df756b-120f-4026-ac1f-b14507717d42-crio-socket\") pod \"insights-runtime-extractor-rhzfq\" (UID: \"29df756b-120f-4026-ac1f-b14507717d42\") " pod="openshift-insights/insights-runtime-extractor-rhzfq" Sep 08 18:24:24.587804 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:24.587789 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/29df756b-120f-4026-ac1f-b14507717d42-kube-rbac-proxy-cm podName:29df756b-120f-4026-ac1f-b14507717d42 nodeName:}" failed. No retries permitted until 2026-09-08 18:24:25.087770299 +0000 UTC m=+47.293755675 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/29df756b-120f-4026-ac1f-b14507717d42-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rhzfq" (UID: "29df756b-120f-4026-ac1f-b14507717d42") : configmap "kube-rbac-proxy" not found Sep 08 18:24:24.587804 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:24.587715 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 08 18:24:24.587953 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:24.587890 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29df756b-120f-4026-ac1f-b14507717d42-insights-runtime-extractor-tls podName:29df756b-120f-4026-ac1f-b14507717d42 nodeName:}" failed. No retries permitted until 2026-09-08 18:24:25.087855582 +0000 UTC m=+47.293840962 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/29df756b-120f-4026-ac1f-b14507717d42-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rhzfq" (UID: "29df756b-120f-4026-ac1f-b14507717d42") : secret "insights-runtime-extractor-tls" not found Sep 08 18:24:24.587953 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:24.587901 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/29df756b-120f-4026-ac1f-b14507717d42-data-volume\") pod \"insights-runtime-extractor-rhzfq\" (UID: \"29df756b-120f-4026-ac1f-b14507717d42\") " pod="openshift-insights/insights-runtime-extractor-rhzfq" Sep 08 18:24:24.599731 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:24.599676 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khbsc\" (UniqueName: \"kubernetes.io/projected/29df756b-120f-4026-ac1f-b14507717d42-kube-api-access-khbsc\") pod \"insights-runtime-extractor-rhzfq\" (UID: \"29df756b-120f-4026-ac1f-b14507717d42\") " pod="openshift-insights/insights-runtime-extractor-rhzfq" Sep 08 18:24:25.092613 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:25.092578 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29df756b-120f-4026-ac1f-b14507717d42-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rhzfq\" (UID: \"29df756b-120f-4026-ac1f-b14507717d42\") " pod="openshift-insights/insights-runtime-extractor-rhzfq" Sep 08 18:24:25.092798 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:25.092627 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/29df756b-120f-4026-ac1f-b14507717d42-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rhzfq\" (UID: \"29df756b-120f-4026-ac1f-b14507717d42\") " pod="openshift-insights/insights-runtime-extractor-rhzfq" Sep 08 18:24:25.092798 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:25.092755 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 08 18:24:25.092798 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:25.092766 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 08 18:24:25.092900 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:25.092852 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/29df756b-120f-4026-ac1f-b14507717d42-kube-rbac-proxy-cm podName:29df756b-120f-4026-ac1f-b14507717d42 nodeName:}" failed. No retries permitted until 2026-09-08 18:24:26.09283461 +0000 UTC m=+48.298820005 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/29df756b-120f-4026-ac1f-b14507717d42-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rhzfq" (UID: "29df756b-120f-4026-ac1f-b14507717d42") : configmap "kube-rbac-proxy" not found Sep 08 18:24:25.092900 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:25.092867 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29df756b-120f-4026-ac1f-b14507717d42-insights-runtime-extractor-tls podName:29df756b-120f-4026-ac1f-b14507717d42 nodeName:}" failed. No retries permitted until 2026-09-08 18:24:26.092861321 +0000 UTC m=+48.298846698 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/29df756b-120f-4026-ac1f-b14507717d42-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rhzfq" (UID: "29df756b-120f-4026-ac1f-b14507717d42") : secret "insights-runtime-extractor-tls" not found Sep 08 18:24:25.223323 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:25.223293 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:25.323626 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:25.323582 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:25.559392 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:25.559366 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9n28f_911c61b8-766b-4ae8-b046-784e22e85889/console-operator/1.log" Sep 08 18:24:25.560006 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:25.559677 2555 scope.go:117] "RemoveContainer" containerID="601b310cdcd96939941a23b18faf640731aac9e5dca5a1b478cf85f83d9c1d2e" Sep 08 18:24:25.560006 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:25.559870 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-9n28f_openshift-console-operator(911c61b8-766b-4ae8-b046-784e22e85889)\"" pod="openshift-console-operator/console-operator-85b454874b-9n28f" podUID="911c61b8-766b-4ae8-b046-784e22e85889" Sep 08 18:24:25.978133 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:25.978052 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:26.030280 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:26.030251 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:26.101729 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:26.101678 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29df756b-120f-4026-ac1f-b14507717d42-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rhzfq\" (UID: \"29df756b-120f-4026-ac1f-b14507717d42\") " pod="openshift-insights/insights-runtime-extractor-rhzfq" Sep 08 18:24:26.101886 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:26.101743 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/29df756b-120f-4026-ac1f-b14507717d42-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rhzfq\" (UID: \"29df756b-120f-4026-ac1f-b14507717d42\") " pod="openshift-insights/insights-runtime-extractor-rhzfq" Sep 08 18:24:26.101886 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:26.101830 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 08 18:24:26.101886 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:26.101832 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 08 18:24:26.101886 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:26.101886 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/29df756b-120f-4026-ac1f-b14507717d42-kube-rbac-proxy-cm podName:29df756b-120f-4026-ac1f-b14507717d42 nodeName:}" failed. No retries permitted until 2026-09-08 18:24:28.101871521 +0000 UTC m=+50.307856900 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/29df756b-120f-4026-ac1f-b14507717d42-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rhzfq" (UID: "29df756b-120f-4026-ac1f-b14507717d42") : configmap "kube-rbac-proxy" not found Sep 08 18:24:26.102072 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:26.101900 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29df756b-120f-4026-ac1f-b14507717d42-insights-runtime-extractor-tls podName:29df756b-120f-4026-ac1f-b14507717d42 nodeName:}" failed. No retries permitted until 2026-09-08 18:24:28.101894292 +0000 UTC m=+50.307879668 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/29df756b-120f-4026-ac1f-b14507717d42-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rhzfq" (UID: "29df756b-120f-4026-ac1f-b14507717d42") : secret "insights-runtime-extractor-tls" not found Sep 08 18:24:26.178971 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:26.178937 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:26.248784 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:26.248712 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lnhcj_582b7b23-efed-41a0-a58f-726efcdff3a8/dns-node-resolver/0.log" Sep 08 18:24:26.279263 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:26.279225 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:26.425362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:26.425330 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:26.505497 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:26.505401 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c2d77fcf-b01d-4341-8978-41cda35ebe81-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-8pt6l\" (UID: \"c2d77fcf-b01d-4341-8978-41cda35ebe81\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8pt6l" Sep 08 18:24:26.505653 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:26.505549 2555 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 08 18:24:26.505653 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:26.505609 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2d77fcf-b01d-4341-8978-41cda35ebe81-samples-operator-tls podName:c2d77fcf-b01d-4341-8978-41cda35ebe81 nodeName:}" failed. No retries permitted until 2026-09-08 18:24:34.505595382 +0000 UTC m=+56.711580757 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c2d77fcf-b01d-4341-8978-41cda35ebe81-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-8pt6l" (UID: "c2d77fcf-b01d-4341-8978-41cda35ebe81") : secret "samples-operator-tls" not found Sep 08 18:24:26.530250 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:26.530215 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:26.590336 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:26.590303 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:26.606494 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:26.606461 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c44b4985-58c8-4e47-af99-0fd9052f8c7b-service-ca-bundle\") pod \"router-default-7bdcf569fd-2mtjd\" (UID: \"c44b4985-58c8-4e47-af99-0fd9052f8c7b\") " pod="openshift-ingress/router-default-7bdcf569fd-2mtjd" Sep 08 18:24:26.606596 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:26.606499 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c44b4985-58c8-4e47-af99-0fd9052f8c7b-metrics-certs\") pod \"router-default-7bdcf569fd-2mtjd\" (UID: \"c44b4985-58c8-4e47-af99-0fd9052f8c7b\") " pod="openshift-ingress/router-default-7bdcf569fd-2mtjd" Sep 08 18:24:26.606632 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:26.606599 2555 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 08 18:24:26.606674 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:26.606633 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c44b4985-58c8-4e47-af99-0fd9052f8c7b-service-ca-bundle podName:c44b4985-58c8-4e47-af99-0fd9052f8c7b nodeName:}" failed. No retries permitted until 2026-09-08 18:24:34.606616058 +0000 UTC m=+56.812601433 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c44b4985-58c8-4e47-af99-0fd9052f8c7b-service-ca-bundle") pod "router-default-7bdcf569fd-2mtjd" (UID: "c44b4985-58c8-4e47-af99-0fd9052f8c7b") : configmap references non-existent config key: service-ca.crt Sep 08 18:24:26.606674 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:26.606652 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c44b4985-58c8-4e47-af99-0fd9052f8c7b-metrics-certs podName:c44b4985-58c8-4e47-af99-0fd9052f8c7b nodeName:}" failed. No retries permitted until 2026-09-08 18:24:34.606645872 +0000 UTC m=+56.812631247 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c44b4985-58c8-4e47-af99-0fd9052f8c7b-metrics-certs") pod "router-default-7bdcf569fd-2mtjd" (UID: "c44b4985-58c8-4e47-af99-0fd9052f8c7b") : secret "router-metrics-certs-default" not found Sep 08 18:24:26.684635 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:26.684575 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:26.786386 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:26.786306 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:27.049161 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:27.049136 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dwsrv_e5d99ca2-78e9-43bb-946d-a944ffd92d62/node-ca/0.log" Sep 08 18:24:28.049112 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:28.049085 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-k4dxz_a067ecec-0c84-4299-812c-6441d126104f/serve-healthcheck-canary/0.log" Sep 08 18:24:28.121423 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:28.121385 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29df756b-120f-4026-ac1f-b14507717d42-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rhzfq\" (UID: \"29df756b-120f-4026-ac1f-b14507717d42\") " pod="openshift-insights/insights-runtime-extractor-rhzfq" Sep 08 18:24:28.121604 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:28.121446 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/29df756b-120f-4026-ac1f-b14507717d42-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rhzfq\" (UID: \"29df756b-120f-4026-ac1f-b14507717d42\") " pod="openshift-insights/insights-runtime-extractor-rhzfq" Sep 08 18:24:28.121604 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:28.121554 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 08 18:24:28.121604 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:28.121579 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 08 18:24:28.121741 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:28.121622 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29df756b-120f-4026-ac1f-b14507717d42-insights-runtime-extractor-tls podName:29df756b-120f-4026-ac1f-b14507717d42 nodeName:}" failed. No retries permitted until 2026-09-08 18:24:32.121606464 +0000 UTC m=+54.327591840 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/29df756b-120f-4026-ac1f-b14507717d42-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rhzfq" (UID: "29df756b-120f-4026-ac1f-b14507717d42") : secret "insights-runtime-extractor-tls" not found Sep 08 18:24:28.121741 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:28.121636 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/29df756b-120f-4026-ac1f-b14507717d42-kube-rbac-proxy-cm podName:29df756b-120f-4026-ac1f-b14507717d42 nodeName:}" failed. No retries permitted until 2026-09-08 18:24:32.121629948 +0000 UTC m=+54.327615324 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/29df756b-120f-4026-ac1f-b14507717d42-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rhzfq" (UID: "29df756b-120f-4026-ac1f-b14507717d42") : configmap "kube-rbac-proxy" not found Sep 08 18:24:28.382810 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:28.382737 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:28.440397 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:28.440364 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:28.524822 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:28.524784 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9de9b23a-f1e9-4b44-87a2-680997e061f0-metrics-tls\") pod \"dns-default-pzzmm\" (UID: \"9de9b23a-f1e9-4b44-87a2-680997e061f0\") " pod="openshift-dns/dns-default-pzzmm" Sep 08 18:24:28.525059 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:28.524839 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/612d5222-58c3-4da0-adc4-760f3fcd1843-registry-tls\") pod \"image-registry-6c87577785-px7mf\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " pod="openshift-image-registry/image-registry-6c87577785-px7mf" Sep 08 18:24:28.525059 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:28.524950 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 08 18:24:28.525059 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:28.524999 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 08 18:24:28.525059 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:28.525010 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c87577785-px7mf: secret "image-registry-tls" not found Sep 08 18:24:28.525059 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:28.525019 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9de9b23a-f1e9-4b44-87a2-680997e061f0-metrics-tls podName:9de9b23a-f1e9-4b44-87a2-680997e061f0 nodeName:}" failed. No retries permitted until 2026-09-08 18:24:44.525001457 +0000 UTC m=+66.730986859 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9de9b23a-f1e9-4b44-87a2-680997e061f0-metrics-tls") pod "dns-default-pzzmm" (UID: "9de9b23a-f1e9-4b44-87a2-680997e061f0") : secret "dns-default-metrics-tls" not found Sep 08 18:24:28.525059 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:28.525045 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/612d5222-58c3-4da0-adc4-760f3fcd1843-registry-tls podName:612d5222-58c3-4da0-adc4-760f3fcd1843 nodeName:}" failed. No retries permitted until 2026-09-08 18:24:44.525034922 +0000 UTC m=+66.731020298 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/612d5222-58c3-4da0-adc4-760f3fcd1843-registry-tls") pod "image-registry-6c87577785-px7mf" (UID: "612d5222-58c3-4da0-adc4-760f3fcd1843") : secret "image-registry-tls" not found Sep 08 18:24:29.124782 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:29.124731 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-9n28f" Sep 08 18:24:29.125223 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:29.124808 2555 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-9n28f" Sep 08 18:24:29.125223 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:29.125131 2555 scope.go:117] "RemoveContainer" containerID="601b310cdcd96939941a23b18faf640731aac9e5dca5a1b478cf85f83d9c1d2e" Sep 08 18:24:29.125299 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:29.125277 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-9n28f_openshift-console-operator(911c61b8-766b-4ae8-b046-784e22e85889)\"" pod="openshift-console-operator/console-operator-85b454874b-9n28f" podUID="911c61b8-766b-4ae8-b046-784e22e85889" Sep 08 18:24:29.218498 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:29.218471 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:31.765150 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:31.765114 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:32.152611 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:32.152560 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29df756b-120f-4026-ac1f-b14507717d42-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rhzfq\" (UID: \"29df756b-120f-4026-ac1f-b14507717d42\") " pod="openshift-insights/insights-runtime-extractor-rhzfq" Sep 08 18:24:32.152832 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:32.152634 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/29df756b-120f-4026-ac1f-b14507717d42-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rhzfq\" (UID: \"29df756b-120f-4026-ac1f-b14507717d42\") " pod="openshift-insights/insights-runtime-extractor-rhzfq" Sep 08 18:24:32.152832 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:32.152747 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 08 18:24:32.152832 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:32.152777 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 08 18:24:32.152832 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:32.152809 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29df756b-120f-4026-ac1f-b14507717d42-insights-runtime-extractor-tls podName:29df756b-120f-4026-ac1f-b14507717d42 nodeName:}" failed. No retries permitted until 2026-09-08 18:24:40.152793671 +0000 UTC m=+62.358779047 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/29df756b-120f-4026-ac1f-b14507717d42-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rhzfq" (UID: "29df756b-120f-4026-ac1f-b14507717d42") : secret "insights-runtime-extractor-tls" not found Sep 08 18:24:32.152832 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:32.152834 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/29df756b-120f-4026-ac1f-b14507717d42-kube-rbac-proxy-cm podName:29df756b-120f-4026-ac1f-b14507717d42 nodeName:}" failed. No retries permitted until 2026-09-08 18:24:40.152820447 +0000 UTC m=+62.358805824 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/29df756b-120f-4026-ac1f-b14507717d42-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rhzfq" (UID: "29df756b-120f-4026-ac1f-b14507717d42") : configmap "kube-rbac-proxy" not found Sep 08 18:24:33.126622 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:33.126587 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:33.226380 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:33.226348 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:33.324324 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:33.324289 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:33.623417 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:33.623373 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:34.039271 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:34.039234 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:34.571313 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:34.571271 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c2d77fcf-b01d-4341-8978-41cda35ebe81-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-8pt6l\" (UID: \"c2d77fcf-b01d-4341-8978-41cda35ebe81\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8pt6l" Sep 08 18:24:34.574105 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:34.574080 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c2d77fcf-b01d-4341-8978-41cda35ebe81-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-8pt6l\" (UID: \"c2d77fcf-b01d-4341-8978-41cda35ebe81\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8pt6l" Sep 08 18:24:34.613569 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:34.613530 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8pt6l" Sep 08 18:24:34.619843 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:34.619803 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:34.672491 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:34.672455 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c44b4985-58c8-4e47-af99-0fd9052f8c7b-service-ca-bundle\") pod \"router-default-7bdcf569fd-2mtjd\" (UID: \"c44b4985-58c8-4e47-af99-0fd9052f8c7b\") " pod="openshift-ingress/router-default-7bdcf569fd-2mtjd" Sep 08 18:24:34.672643 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:34.672508 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c44b4985-58c8-4e47-af99-0fd9052f8c7b-metrics-certs\") pod \"router-default-7bdcf569fd-2mtjd\" (UID: \"c44b4985-58c8-4e47-af99-0fd9052f8c7b\") " pod="openshift-ingress/router-default-7bdcf569fd-2mtjd" Sep 08 18:24:34.672741 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:34.672648 2555 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 08 18:24:34.672741 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:34.672653 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c44b4985-58c8-4e47-af99-0fd9052f8c7b-service-ca-bundle podName:c44b4985-58c8-4e47-af99-0fd9052f8c7b nodeName:}" failed. No retries permitted until 2026-09-08 18:24:50.672628728 +0000 UTC m=+72.878614122 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c44b4985-58c8-4e47-af99-0fd9052f8c7b-service-ca-bundle") pod "router-default-7bdcf569fd-2mtjd" (UID: "c44b4985-58c8-4e47-af99-0fd9052f8c7b") : configmap references non-existent config key: service-ca.crt Sep 08 18:24:34.672741 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:34.672739 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c44b4985-58c8-4e47-af99-0fd9052f8c7b-metrics-certs podName:c44b4985-58c8-4e47-af99-0fd9052f8c7b nodeName:}" failed. No retries permitted until 2026-09-08 18:24:50.672721503 +0000 UTC m=+72.878706898 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c44b4985-58c8-4e47-af99-0fd9052f8c7b-metrics-certs") pod "router-default-7bdcf569fd-2mtjd" (UID: "c44b4985-58c8-4e47-af99-0fd9052f8c7b") : secret "router-metrics-certs-default" not found Sep 08 18:24:34.912742 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:34.912637 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8pt6l"] Sep 08 18:24:35.026292 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:35.026256 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:35.417931 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:35.417894 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:35.518763 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:35.518731 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:35.588053 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:35.588022 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8pt6l" event={"ID":"c2d77fcf-b01d-4341-8978-41cda35ebe81","Type":"ContainerStarted","Data":"4ce957a3cc8103cc06089930c4db5e61248c470bb2628fc642bd3057679e576a"} Sep 08 18:24:35.620262 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:35.620225 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:36.220026 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:36.219987 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:36.431896 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:36.431861 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:36.811782 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:36.811741 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:36.914088 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:36.914057 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:37.017228 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:37.017199 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:37.420272 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:37.420183 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:37.495254 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:37.495228 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cc9pd" Sep 08 18:24:37.521461 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:37.521421 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:37.595336 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:37.595302 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8pt6l" event={"ID":"c2d77fcf-b01d-4341-8978-41cda35ebe81","Type":"ContainerStarted","Data":"eea70aa589a4cc83122977f871ca4f542ad326233aa6c32534715faec3a18929"} Sep 08 18:24:37.595494 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:37.595362 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8pt6l" event={"ID":"c2d77fcf-b01d-4341-8978-41cda35ebe81","Type":"ContainerStarted","Data":"8e9990bf4ff0255a1f9200b060258b613dc9a790395d948567fb903a0130a4dd"} Sep 08 18:24:37.610521 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:37.610476 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-8pt6l" podStartSLOduration=17.629802002 podStartE2EDuration="19.610460633s" totalImagesPullingTime="1.980658631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:24:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:24:37.609539855 +0000 UTC m=+59.815525254" watchObservedRunningTime="2026-09-08 18:24:37.610460633 +0000 UTC m=+59.816446030" Sep 08 18:24:37.614167 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:37.614130 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:38.013521 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:38.013483 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:38.818142 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:38.818105 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:39.025007 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:39.024964 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:39.395082 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:39.395047 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:39.506868 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:39.506834 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:39.643794 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:39.643755 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:40.217733 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:40.217677 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29df756b-120f-4026-ac1f-b14507717d42-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rhzfq\" (UID: \"29df756b-120f-4026-ac1f-b14507717d42\") " pod="openshift-insights/insights-runtime-extractor-rhzfq" Sep 08 18:24:40.218128 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:40.217749 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/29df756b-120f-4026-ac1f-b14507717d42-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rhzfq\" (UID: \"29df756b-120f-4026-ac1f-b14507717d42\") " pod="openshift-insights/insights-runtime-extractor-rhzfq" Sep 08 18:24:40.218128 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:40.217801 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 08 18:24:40.218128 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:24:40.217879 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29df756b-120f-4026-ac1f-b14507717d42-insights-runtime-extractor-tls podName:29df756b-120f-4026-ac1f-b14507717d42 nodeName:}" failed. No retries permitted until 2026-09-08 18:24:56.21786368 +0000 UTC m=+78.423849056 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/29df756b-120f-4026-ac1f-b14507717d42-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rhzfq" (UID: "29df756b-120f-4026-ac1f-b14507717d42") : secret "insights-runtime-extractor-tls" not found Sep 08 18:24:40.218261 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:40.218244 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/29df756b-120f-4026-ac1f-b14507717d42-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rhzfq\" (UID: \"29df756b-120f-4026-ac1f-b14507717d42\") " pod="openshift-insights/insights-runtime-extractor-rhzfq" Sep 08 18:24:40.219966 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:40.219943 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:40.318763 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:40.318737 2555 scope.go:117] "RemoveContainer" containerID="601b310cdcd96939941a23b18faf640731aac9e5dca5a1b478cf85f83d9c1d2e" Sep 08 18:24:40.419176 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:40.419139 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:40.606819 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:40.606792 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9n28f_911c61b8-766b-4ae8-b046-784e22e85889/console-operator/1.log" Sep 08 18:24:40.606990 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:40.606840 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-9n28f" event={"ID":"911c61b8-766b-4ae8-b046-784e22e85889","Type":"ContainerStarted","Data":"0b01698933cac141e12836f701ecc3278929e5c059b3c21e3b8841ad87add6ce"} Sep 08 18:24:40.607136 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:40.607113 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-9n28f" Sep 08 18:24:40.621978 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:40.621941 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:40.625426 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:40.625380 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-9n28f" podStartSLOduration=19.116225317 podStartE2EDuration="22.62536601s" totalImagesPullingTime="3.509140693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:24:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:24:40.624390181 +0000 UTC m=+62.830375650" watchObservedRunningTime="2026-09-08 18:24:40.62536601 +0000 UTC m=+62.831351472" Sep 08 18:24:40.768456 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:40.768424 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-9n28f" Sep 08 18:24:40.975658 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:40.975575 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:41.077233 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:41.077198 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:41.133988 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:41.133958 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:41.248982 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:41.248902 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:41.421343 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:41.421312 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:41.829963 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:41.829927 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:41.927878 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:41.927804 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:42.026857 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:42.026825 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:42.082917 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:42.082821 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:42.213403 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:42.213360 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:42.314740 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:42.314681 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:42.420343 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:42.420238 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:42.620033 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:42.620006 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:42.721842 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:42.721734 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:42.836039 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:42.836001 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:43.618553 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:43.618511 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:43.718801 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:43.718754 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:43.818619 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:43.818540 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:44.022141 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:44.022060 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:44.048410 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:44.048372 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2-metrics-certs\") pod \"network-metrics-daemon-qz9vk\" (UID: \"3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2\") " pod="openshift-multus/network-metrics-daemon-qz9vk" Sep 08 18:24:44.048501 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:44.048423 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2744e789-a69b-4c63-b2db-2a7676c746ca-original-pull-secret\") pod \"global-pull-secret-syncer-qbksq\" (UID: \"2744e789-a69b-4c63-b2db-2a7676c746ca\") " pod="kube-system/global-pull-secret-syncer-qbksq" Sep 08 18:24:44.050833 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:44.050813 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 08 18:24:44.051317 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:44.051299 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 08 18:24:44.061218 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:44.061194 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2744e789-a69b-4c63-b2db-2a7676c746ca-original-pull-secret\") pod \"global-pull-secret-syncer-qbksq\" (UID: \"2744e789-a69b-4c63-b2db-2a7676c746ca\") " pod="kube-system/global-pull-secret-syncer-qbksq" Sep 08 18:24:44.061350 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:44.061238 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2-metrics-certs\") pod \"network-metrics-daemon-qz9vk\" (UID: \"3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2\") " pod="openshift-multus/network-metrics-daemon-qz9vk" Sep 08 18:24:44.218593 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:44.218565 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:44.231522 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:44.231487 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-frqcg\"" Sep 08 18:24:44.236445 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:44.236424 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qbksq" Sep 08 18:24:44.240542 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:44.240518 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qz9vk" Sep 08 18:24:44.250857 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:44.250833 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mdvx\" (UniqueName: \"kubernetes.io/projected/deed349e-43fc-4702-aef1-b9334795fd8e-kube-api-access-9mdvx\") pod \"network-check-target-tqthm\" (UID: \"deed349e-43fc-4702-aef1-b9334795fd8e\") " pod="openshift-network-diagnostics/network-check-target-tqthm" Sep 08 18:24:44.253894 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:44.253870 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 08 18:24:44.263652 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:44.263627 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 08 18:24:44.274793 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:44.274723 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mdvx\" (UniqueName: \"kubernetes.io/projected/deed349e-43fc-4702-aef1-b9334795fd8e-kube-api-access-9mdvx\") pod \"network-check-target-tqthm\" (UID: \"deed349e-43fc-4702-aef1-b9334795fd8e\") " pod="openshift-network-diagnostics/network-check-target-tqthm" Sep 08 18:24:44.512177 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:44.512142 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qbksq"] Sep 08 18:24:44.515517 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:24:44.515486 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2744e789_a69b_4c63_b2db_2a7676c746ca.slice/crio-6436b3925531297fb267766f898efe248582ecef012f895c217aab9984d46519 WatchSource:0}: Error finding container 6436b3925531297fb267766f898efe248582ecef012f895c217aab9984d46519: Status 404 returned error can't find the container with id 6436b3925531297fb267766f898efe248582ecef012f895c217aab9984d46519 Sep 08 18:24:44.515651 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:44.515527 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qz9vk"] Sep 08 18:24:44.518368 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:24:44.518349 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d5d3a1e_2560_49ce_8f2f_c0e8cdde2ec2.slice/crio-1bd15b5bfe16905b18e4e762c47dbbdb7a5c9f9f4f7994153e82c1ac8b2f5114 WatchSource:0}: Error finding container 1bd15b5bfe16905b18e4e762c47dbbdb7a5c9f9f4f7994153e82c1ac8b2f5114: Status 404 returned error can't find the container with id 1bd15b5bfe16905b18e4e762c47dbbdb7a5c9f9f4f7994153e82c1ac8b2f5114 Sep 08 18:24:44.545080 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:44.545052 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jmlkr\"" Sep 08 18:24:44.553014 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:44.552987 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9de9b23a-f1e9-4b44-87a2-680997e061f0-metrics-tls\") pod \"dns-default-pzzmm\" (UID: \"9de9b23a-f1e9-4b44-87a2-680997e061f0\") " pod="openshift-dns/dns-default-pzzmm" Sep 08 18:24:44.553145 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:44.553027 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/612d5222-58c3-4da0-adc4-760f3fcd1843-registry-tls\") pod \"image-registry-6c87577785-px7mf\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " pod="openshift-image-registry/image-registry-6c87577785-px7mf" Sep 08 18:24:44.553145 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:44.553102 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tqthm" Sep 08 18:24:44.555432 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:44.555413 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9de9b23a-f1e9-4b44-87a2-680997e061f0-metrics-tls\") pod \"dns-default-pzzmm\" (UID: \"9de9b23a-f1e9-4b44-87a2-680997e061f0\") " pod="openshift-dns/dns-default-pzzmm" Sep 08 18:24:44.555531 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:44.555479 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/612d5222-58c3-4da0-adc4-760f3fcd1843-registry-tls\") pod \"image-registry-6c87577785-px7mf\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " pod="openshift-image-registry/image-registry-6c87577785-px7mf" Sep 08 18:24:44.622297 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:44.622266 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qz9vk" event={"ID":"3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2","Type":"ContainerStarted","Data":"1bd15b5bfe16905b18e4e762c47dbbdb7a5c9f9f4f7994153e82c1ac8b2f5114"} Sep 08 18:24:44.623514 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:44.623482 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qbksq" event={"ID":"2744e789-a69b-4c63-b2db-2a7676c746ca","Type":"ContainerStarted","Data":"6436b3925531297fb267766f898efe248582ecef012f895c217aab9984d46519"} Sep 08 18:24:44.777935 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:44.777899 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-cm2c5\"" Sep 08 18:24:44.786513 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:44.786493 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c87577785-px7mf" Sep 08 18:24:44.805868 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:44.805846 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vbqmk\"" Sep 08 18:24:44.814604 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:44.814578 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pzzmm" Sep 08 18:24:44.820382 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:44.820360 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tqthm"] Sep 08 18:24:44.824418 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:24:44.824392 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddeed349e_43fc_4702_aef1_b9334795fd8e.slice/crio-5ea7a17c184b26db9e6c4d8eb0ddb60992f49d74ef12963cc727c67faa43f28a WatchSource:0}: Error finding container 5ea7a17c184b26db9e6c4d8eb0ddb60992f49d74ef12963cc727c67faa43f28a: Status 404 returned error can't find the container with id 5ea7a17c184b26db9e6c4d8eb0ddb60992f49d74ef12963cc727c67faa43f28a Sep 08 18:24:45.066278 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:45.066205 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c87577785-px7mf"] Sep 08 18:24:45.096831 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:45.096803 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pzzmm"] Sep 08 18:24:45.101038 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:24:45.101013 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9de9b23a_f1e9_4b44_87a2_680997e061f0.slice/crio-6fab1a53b199f6ad838988fca9ff969847464fa4dad25d243f469e1cc03ff95d WatchSource:0}: Error finding container 6fab1a53b199f6ad838988fca9ff969847464fa4dad25d243f469e1cc03ff95d: Status 404 returned error can't find the container with id 6fab1a53b199f6ad838988fca9ff969847464fa4dad25d243f469e1cc03ff95d Sep 08 18:24:45.637349 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:45.637312 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tqthm" event={"ID":"deed349e-43fc-4702-aef1-b9334795fd8e","Type":"ContainerStarted","Data":"5ea7a17c184b26db9e6c4d8eb0ddb60992f49d74ef12963cc727c67faa43f28a"} Sep 08 18:24:45.639481 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:45.639447 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pzzmm" event={"ID":"9de9b23a-f1e9-4b44-87a2-680997e061f0","Type":"ContainerStarted","Data":"6fab1a53b199f6ad838988fca9ff969847464fa4dad25d243f469e1cc03ff95d"} Sep 08 18:24:45.641530 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:45.641501 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c87577785-px7mf" event={"ID":"612d5222-58c3-4da0-adc4-760f3fcd1843","Type":"ContainerStarted","Data":"af065103851dab2814d45c86c5c04d6793f8ab5ab02346eb3bcf00e961fc0fd8"} Sep 08 18:24:45.641654 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:45.641552 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c87577785-px7mf" event={"ID":"612d5222-58c3-4da0-adc4-760f3fcd1843","Type":"ContainerStarted","Data":"13fd40dc7dccad0e3eeec99a8d668c2ffdedb26a427124940c1594d382017f11"} Sep 08 18:24:45.641826 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:45.641802 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6c87577785-px7mf" Sep 08 18:24:45.665803 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:45.665737 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6c87577785-px7mf" podStartSLOduration=66.665716713 podStartE2EDuration="1m6.665716713s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:23:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:24:45.665360437 +0000 UTC m=+67.871345849" watchObservedRunningTime="2026-09-08 18:24:45.665716713 +0000 UTC m=+67.871702112" Sep 08 18:24:46.649230 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:46.649191 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qz9vk" event={"ID":"3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2","Type":"ContainerStarted","Data":"13decb7b15f1766de3995a587c1412d9e634eb09341884d3a881e0a8fd75047e"} Sep 08 18:24:46.649715 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:46.649253 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qz9vk" event={"ID":"3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2","Type":"ContainerStarted","Data":"8d47c15b9961e54252de48eaef56911712ec17d1537dc33cb995bf3144e03436"} Sep 08 18:24:46.668638 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:46.667839 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-qz9vk" podStartSLOduration=66.569276691 podStartE2EDuration="1m7.667817296s" totalImagesPullingTime="1.098540605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:23:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:24:46.667362144 +0000 UTC m=+68.873347545" watchObservedRunningTime="2026-09-08 18:24:46.667817296 +0000 UTC m=+68.873802696" Sep 08 18:24:47.933490 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:47.933449 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-ctmck"] Sep 08 18:24:47.955461 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:47.955426 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-ctmck"] Sep 08 18:24:47.955656 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:47.955536 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-ctmck" Sep 08 18:24:47.957873 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:47.957844 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 08 18:24:47.958005 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:47.957947 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 08 18:24:47.958005 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:47.957952 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-cg2hq\"" Sep 08 18:24:47.984458 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:47.984413 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:47.985539 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:47.985502 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrgxt\" (UniqueName: \"kubernetes.io/projected/b514b325-9daf-4c8f-a8c1-cee65d51c2f0-kube-api-access-jrgxt\") pod \"downloads-85d98c998f-ctmck\" (UID: \"b514b325-9daf-4c8f-a8c1-cee65d51c2f0\") " pod="openshift-console/downloads-85d98c998f-ctmck" Sep 08 18:24:48.065922 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:48.065894 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:48.086208 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:48.086165 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrgxt\" (UniqueName: \"kubernetes.io/projected/b514b325-9daf-4c8f-a8c1-cee65d51c2f0-kube-api-access-jrgxt\") pod \"downloads-85d98c998f-ctmck\" (UID: \"b514b325-9daf-4c8f-a8c1-cee65d51c2f0\") " pod="openshift-console/downloads-85d98c998f-ctmck" Sep 08 18:24:48.095285 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:48.095257 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrgxt\" (UniqueName: \"kubernetes.io/projected/b514b325-9daf-4c8f-a8c1-cee65d51c2f0-kube-api-access-jrgxt\") pod \"downloads-85d98c998f-ctmck\" (UID: \"b514b325-9daf-4c8f-a8c1-cee65d51c2f0\") " pod="openshift-console/downloads-85d98c998f-ctmck" Sep 08 18:24:48.170395 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:48.170364 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:48.267515 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:48.267415 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-ctmck" Sep 08 18:24:48.268802 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:48.268775 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:50.519507 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:50.519476 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-ctmck"] Sep 08 18:24:50.523105 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:24:50.523076 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb514b325_9daf_4c8f_a8c1_cee65d51c2f0.slice/crio-9e6910364c9cd2230de1513cbfee670a021d1fb81e4ef03a33522e694b89ed8d WatchSource:0}: Error finding container 9e6910364c9cd2230de1513cbfee670a021d1fb81e4ef03a33522e694b89ed8d: Status 404 returned error can't find the container with id 9e6910364c9cd2230de1513cbfee670a021d1fb81e4ef03a33522e694b89ed8d Sep 08 18:24:50.572138 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:50.572106 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:50.665006 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:50.664973 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pzzmm" event={"ID":"9de9b23a-f1e9-4b44-87a2-680997e061f0","Type":"ContainerStarted","Data":"e3159b932e016026eb35868dd14d0a71984c58b694a7fada6115eab969796b73"} Sep 08 18:24:50.665208 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:50.665029 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pzzmm" event={"ID":"9de9b23a-f1e9-4b44-87a2-680997e061f0","Type":"ContainerStarted","Data":"37c084fdd8382755eddaf0ccd9556491761c502a6cc9c680517d8fa3f497e980"} Sep 08 18:24:50.665208 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:50.665068 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-pzzmm" Sep 08 18:24:50.666237 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:50.666202 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-ctmck" event={"ID":"b514b325-9daf-4c8f-a8c1-cee65d51c2f0","Type":"ContainerStarted","Data":"9e6910364c9cd2230de1513cbfee670a021d1fb81e4ef03a33522e694b89ed8d"} Sep 08 18:24:50.667574 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:50.667554 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qbksq" event={"ID":"2744e789-a69b-4c63-b2db-2a7676c746ca","Type":"ContainerStarted","Data":"968159593e08977196d358018745cffc5c2799cf84d8f43183bbc1196d1798a1"} Sep 08 18:24:50.668992 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:50.668970 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tqthm" event={"ID":"deed349e-43fc-4702-aef1-b9334795fd8e","Type":"ContainerStarted","Data":"77d76d5021df6dda637fd90fb3a8fa8d74998da20a5f334f4fa52fb7cda499d3"} Sep 08 18:24:50.669109 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:50.669098 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-tqthm" Sep 08 18:24:50.673333 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:50.673313 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:50.684960 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:50.684912 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-pzzmm" podStartSLOduration=33.564398177 podStartE2EDuration="38.684897852s" totalImagesPullingTime="5.120499675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:24:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:24:50.68417871 +0000 UTC m=+72.890164100" watchObservedRunningTime="2026-09-08 18:24:50.684897852 +0000 UTC m=+72.890883301" Sep 08 18:24:50.700032 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:50.699981 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-qbksq" podStartSLOduration=65.976881426 podStartE2EDuration="1m11.699963691s" totalImagesPullingTime="5.723082265s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 18:23:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:24:50.69895228 +0000 UTC m=+72.904937677" watchObservedRunningTime="2026-09-08 18:24:50.699963691 +0000 UTC m=+72.905949126" Sep 08 18:24:50.707614 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:50.707581 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c44b4985-58c8-4e47-af99-0fd9052f8c7b-service-ca-bundle\") pod \"router-default-7bdcf569fd-2mtjd\" (UID: \"c44b4985-58c8-4e47-af99-0fd9052f8c7b\") " pod="openshift-ingress/router-default-7bdcf569fd-2mtjd" Sep 08 18:24:50.707840 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:50.707816 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c44b4985-58c8-4e47-af99-0fd9052f8c7b-metrics-certs\") pod \"router-default-7bdcf569fd-2mtjd\" (UID: \"c44b4985-58c8-4e47-af99-0fd9052f8c7b\") " pod="openshift-ingress/router-default-7bdcf569fd-2mtjd" Sep 08 18:24:50.708295 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:50.708273 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c44b4985-58c8-4e47-af99-0fd9052f8c7b-service-ca-bundle\") pod \"router-default-7bdcf569fd-2mtjd\" (UID: \"c44b4985-58c8-4e47-af99-0fd9052f8c7b\") " pod="openshift-ingress/router-default-7bdcf569fd-2mtjd" Sep 08 18:24:50.710341 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:50.710321 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c44b4985-58c8-4e47-af99-0fd9052f8c7b-metrics-certs\") pod \"router-default-7bdcf569fd-2mtjd\" (UID: \"c44b4985-58c8-4e47-af99-0fd9052f8c7b\") " pod="openshift-ingress/router-default-7bdcf569fd-2mtjd" Sep 08 18:24:50.713636 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:50.713581 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-tqthm" podStartSLOduration=66.298906436 podStartE2EDuration="1m11.713567328s" totalImagesPullingTime="5.414660892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:23:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:24:50.713355887 +0000 UTC m=+72.919341276" watchObservedRunningTime="2026-09-08 18:24:50.713567328 +0000 UTC m=+72.919552725" Sep 08 18:24:50.947436 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:50.947406 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-6g2gs\"" Sep 08 18:24:50.955318 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:50.955293 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7bdcf569fd-2mtjd" Sep 08 18:24:51.084072 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:51.084041 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:51.172767 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:51.172732 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:51.237445 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:51.237358 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7bdcf569fd-2mtjd"] Sep 08 18:24:51.243328 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:24:51.243298 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc44b4985_58c8_4e47_af99_0fd9052f8c7b.slice/crio-d872838a606b8688b21e138ec0e00b68830d40dabf4464f7754fd133b3339713 WatchSource:0}: Error finding container d872838a606b8688b21e138ec0e00b68830d40dabf4464f7754fd133b3339713: Status 404 returned error can't find the container with id d872838a606b8688b21e138ec0e00b68830d40dabf4464f7754fd133b3339713 Sep 08 18:24:51.675470 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:51.675437 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7bdcf569fd-2mtjd" event={"ID":"c44b4985-58c8-4e47-af99-0fd9052f8c7b","Type":"ContainerStarted","Data":"982642bf848b7c85e6c1d6983c6fa0d58931db9b061e6b787c680843dbba8427"} Sep 08 18:24:51.675948 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:51.675494 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7bdcf569fd-2mtjd" event={"ID":"c44b4985-58c8-4e47-af99-0fd9052f8c7b","Type":"ContainerStarted","Data":"d872838a606b8688b21e138ec0e00b68830d40dabf4464f7754fd133b3339713"} Sep 08 18:24:51.695850 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:51.695797 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7bdcf569fd-2mtjd" podStartSLOduration=33.695775731 podStartE2EDuration="33.695775731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:24:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:24:51.695027444 +0000 UTC m=+73.901012843" watchObservedRunningTime="2026-09-08 18:24:51.695775731 +0000 UTC m=+73.901761132" Sep 08 18:24:51.956473 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:51.956376 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7bdcf569fd-2mtjd" Sep 08 18:24:51.959469 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:51.959443 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7bdcf569fd-2mtjd" Sep 08 18:24:52.112571 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:52.112530 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:52.182987 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:52.182952 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:52.679624 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:52.679591 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7bdcf569fd-2mtjd" Sep 08 18:24:52.681072 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:52.681052 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7bdcf569fd-2mtjd" Sep 08 18:24:54.628209 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:54.628175 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:54.728007 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:54.727957 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:54.780678 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:54.780645 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:55.613806 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:55.613770 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:55.715263 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:55.715228 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:55.889206 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:55.889134 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:55.989212 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:55.989179 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:56.071988 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:56.071957 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:56.262438 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:56.262352 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29df756b-120f-4026-ac1f-b14507717d42-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rhzfq\" (UID: \"29df756b-120f-4026-ac1f-b14507717d42\") " pod="openshift-insights/insights-runtime-extractor-rhzfq" Sep 08 18:24:56.265220 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:56.265193 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29df756b-120f-4026-ac1f-b14507717d42-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rhzfq\" (UID: \"29df756b-120f-4026-ac1f-b14507717d42\") " pod="openshift-insights/insights-runtime-extractor-rhzfq" Sep 08 18:24:56.424483 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:56.424452 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-srf5g\"" Sep 08 18:24:56.433294 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:56.433267 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rhzfq" Sep 08 18:24:56.593905 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:56.593873 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:56.713037 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:56.713009 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rhzfq"] Sep 08 18:24:56.717715 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:24:56.717423 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29df756b_120f_4026_ac1f_b14507717d42.slice/crio-7d39e0f504cf4bd9819d1c428fc4b31f52cb4fca206009276aecf4387e398e6d WatchSource:0}: Error finding container 7d39e0f504cf4bd9819d1c428fc4b31f52cb4fca206009276aecf4387e398e6d: Status 404 returned error can't find the container with id 7d39e0f504cf4bd9819d1c428fc4b31f52cb4fca206009276aecf4387e398e6d Sep 08 18:24:57.701287 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:57.701255 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rhzfq" event={"ID":"29df756b-120f-4026-ac1f-b14507717d42","Type":"ContainerStarted","Data":"c9888f99449db782d6cc83771d72582907fbd4fd5f2912486e11c0b9266c0a13"} Sep 08 18:24:57.701398 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:57.701362 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rhzfq" event={"ID":"29df756b-120f-4026-ac1f-b14507717d42","Type":"ContainerStarted","Data":"7d39e0f504cf4bd9819d1c428fc4b31f52cb4fca206009276aecf4387e398e6d"} Sep 08 18:24:58.707843 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:58.707806 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rhzfq" event={"ID":"29df756b-120f-4026-ac1f-b14507717d42","Type":"ContainerStarted","Data":"335872d4bd6ad5bfae3a906a2741e226a7f35feb9bd5b7d35e0cace7649d2f40"} Sep 08 18:24:58.858231 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:58.858195 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:58.914124 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:58.914041 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:59.571387 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:59.571347 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:59.634360 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:59.634326 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:24:59.713939 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:59.713900 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rhzfq" event={"ID":"29df756b-120f-4026-ac1f-b14507717d42","Type":"ContainerStarted","Data":"d7d6d11d55fa187adb84ec9cf6ef49d6ba92413d2c76bac7558b8a76e5210679"} Sep 08 18:24:59.732247 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:24:59.732191 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-rhzfq" podStartSLOduration=33.358588602 podStartE2EDuration="35.732169343s" totalImagesPullingTime="2.373580741s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 18:24:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:24:59.731069969 +0000 UTC m=+81.937055588" watchObservedRunningTime="2026-09-08 18:24:59.732169343 +0000 UTC m=+81.938154744" Sep 08 18:25:00.094042 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:00.094004 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:00.127323 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:00.127291 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-ggzxh"] Sep 08 18:25:00.130773 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:00.130751 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-ggzxh" Sep 08 18:25:00.135848 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:00.135823 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-6xpj4\"" Sep 08 18:25:00.136181 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:00.136151 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 08 18:25:00.136857 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:00.136838 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 08 18:25:00.137044 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:00.137030 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 08 18:25:00.137122 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:00.137085 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 08 18:25:00.137175 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:00.137142 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 08 18:25:00.149416 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:00.149393 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-ggzxh"] Sep 08 18:25:00.193032 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:00.192971 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1235e212-ee01-4797-8566-fc2b904316fd-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-ggzxh\" (UID: \"1235e212-ee01-4797-8566-fc2b904316fd\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ggzxh" Sep 08 18:25:00.193032 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:00.193022 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1235e212-ee01-4797-8566-fc2b904316fd-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-ggzxh\" (UID: \"1235e212-ee01-4797-8566-fc2b904316fd\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ggzxh" Sep 08 18:25:00.193266 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:00.193098 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zf5kt\" (UniqueName: \"kubernetes.io/projected/1235e212-ee01-4797-8566-fc2b904316fd-kube-api-access-zf5kt\") pod \"prometheus-operator-6f4946db56-ggzxh\" (UID: \"1235e212-ee01-4797-8566-fc2b904316fd\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ggzxh" Sep 08 18:25:00.193266 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:00.193161 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1235e212-ee01-4797-8566-fc2b904316fd-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-ggzxh\" (UID: \"1235e212-ee01-4797-8566-fc2b904316fd\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ggzxh" Sep 08 18:25:00.203929 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:00.203898 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:00.294167 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:00.294125 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1235e212-ee01-4797-8566-fc2b904316fd-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-ggzxh\" (UID: \"1235e212-ee01-4797-8566-fc2b904316fd\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ggzxh" Sep 08 18:25:00.294360 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:00.294223 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1235e212-ee01-4797-8566-fc2b904316fd-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-ggzxh\" (UID: \"1235e212-ee01-4797-8566-fc2b904316fd\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ggzxh" Sep 08 18:25:00.294360 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:00.294249 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1235e212-ee01-4797-8566-fc2b904316fd-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-ggzxh\" (UID: \"1235e212-ee01-4797-8566-fc2b904316fd\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ggzxh" Sep 08 18:25:00.294360 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:00.294273 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zf5kt\" (UniqueName: \"kubernetes.io/projected/1235e212-ee01-4797-8566-fc2b904316fd-kube-api-access-zf5kt\") pod \"prometheus-operator-6f4946db56-ggzxh\" (UID: \"1235e212-ee01-4797-8566-fc2b904316fd\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ggzxh" Sep 08 18:25:00.295512 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:00.295482 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1235e212-ee01-4797-8566-fc2b904316fd-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-ggzxh\" (UID: \"1235e212-ee01-4797-8566-fc2b904316fd\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ggzxh" Sep 08 18:25:00.297016 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:00.296989 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1235e212-ee01-4797-8566-fc2b904316fd-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-ggzxh\" (UID: \"1235e212-ee01-4797-8566-fc2b904316fd\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ggzxh" Sep 08 18:25:00.297122 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:00.296994 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1235e212-ee01-4797-8566-fc2b904316fd-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-ggzxh\" (UID: \"1235e212-ee01-4797-8566-fc2b904316fd\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ggzxh" Sep 08 18:25:00.310827 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:00.310797 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zf5kt\" (UniqueName: \"kubernetes.io/projected/1235e212-ee01-4797-8566-fc2b904316fd-kube-api-access-zf5kt\") pod \"prometheus-operator-6f4946db56-ggzxh\" (UID: \"1235e212-ee01-4797-8566-fc2b904316fd\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-ggzxh" Sep 08 18:25:00.359255 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:00.359149 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:00.441587 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:00.441549 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-ggzxh" Sep 08 18:25:00.461397 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:00.461303 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:00.678456 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:00.678375 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-pzzmm" Sep 08 18:25:00.722347 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:00.722317 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-ggzxh"] Sep 08 18:25:00.725836 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:25:00.725804 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1235e212_ee01_4797_8566_fc2b904316fd.slice/crio-78edddda722c66407db57e84546d805a3caa31c3e7c08066102da09d9509c734 WatchSource:0}: Error finding container 78edddda722c66407db57e84546d805a3caa31c3e7c08066102da09d9509c734: Status 404 returned error can't find the container with id 78edddda722c66407db57e84546d805a3caa31c3e7c08066102da09d9509c734 Sep 08 18:25:01.723002 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:01.722961 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-ggzxh" event={"ID":"1235e212-ee01-4797-8566-fc2b904316fd","Type":"ContainerStarted","Data":"78edddda722c66407db57e84546d805a3caa31c3e7c08066102da09d9509c734"} Sep 08 18:25:03.453001 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:03.452966 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-64d976c5fb-v7g8t"] Sep 08 18:25:03.457277 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:03.457256 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64d976c5fb-v7g8t" Sep 08 18:25:03.459258 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:03.459238 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 08 18:25:03.459952 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:03.459933 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 08 18:25:03.460035 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:03.459975 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 08 18:25:03.460086 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:03.460037 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-vdqzb\"" Sep 08 18:25:03.460086 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:03.459932 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 08 18:25:03.460214 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:03.460194 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 08 18:25:03.466158 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:03.466136 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 08 18:25:03.467138 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:03.466791 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64d976c5fb-v7g8t"] Sep 08 18:25:03.521230 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:03.521194 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b30c08ad-dcd8-400b-8943-437f2d500b1a-service-ca\") pod \"console-64d976c5fb-v7g8t\" (UID: \"b30c08ad-dcd8-400b-8943-437f2d500b1a\") " pod="openshift-console/console-64d976c5fb-v7g8t" Sep 08 18:25:03.521427 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:03.521238 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cz58h\" (UniqueName: \"kubernetes.io/projected/b30c08ad-dcd8-400b-8943-437f2d500b1a-kube-api-access-cz58h\") pod \"console-64d976c5fb-v7g8t\" (UID: \"b30c08ad-dcd8-400b-8943-437f2d500b1a\") " pod="openshift-console/console-64d976c5fb-v7g8t" Sep 08 18:25:03.521427 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:03.521268 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b30c08ad-dcd8-400b-8943-437f2d500b1a-console-oauth-config\") pod \"console-64d976c5fb-v7g8t\" (UID: \"b30c08ad-dcd8-400b-8943-437f2d500b1a\") " pod="openshift-console/console-64d976c5fb-v7g8t" Sep 08 18:25:03.521427 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:03.521349 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b30c08ad-dcd8-400b-8943-437f2d500b1a-trusted-ca-bundle\") pod \"console-64d976c5fb-v7g8t\" (UID: \"b30c08ad-dcd8-400b-8943-437f2d500b1a\") " pod="openshift-console/console-64d976c5fb-v7g8t" Sep 08 18:25:03.521427 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:03.521380 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b30c08ad-dcd8-400b-8943-437f2d500b1a-console-serving-cert\") pod \"console-64d976c5fb-v7g8t\" (UID: \"b30c08ad-dcd8-400b-8943-437f2d500b1a\") " pod="openshift-console/console-64d976c5fb-v7g8t" Sep 08 18:25:03.521639 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:03.521433 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b30c08ad-dcd8-400b-8943-437f2d500b1a-oauth-serving-cert\") pod \"console-64d976c5fb-v7g8t\" (UID: \"b30c08ad-dcd8-400b-8943-437f2d500b1a\") " pod="openshift-console/console-64d976c5fb-v7g8t" Sep 08 18:25:03.521639 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:03.521456 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b30c08ad-dcd8-400b-8943-437f2d500b1a-console-config\") pod \"console-64d976c5fb-v7g8t\" (UID: \"b30c08ad-dcd8-400b-8943-437f2d500b1a\") " pod="openshift-console/console-64d976c5fb-v7g8t" Sep 08 18:25:03.622549 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:03.622514 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b30c08ad-dcd8-400b-8943-437f2d500b1a-oauth-serving-cert\") pod \"console-64d976c5fb-v7g8t\" (UID: \"b30c08ad-dcd8-400b-8943-437f2d500b1a\") " pod="openshift-console/console-64d976c5fb-v7g8t" Sep 08 18:25:03.622754 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:03.622557 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b30c08ad-dcd8-400b-8943-437f2d500b1a-console-config\") pod \"console-64d976c5fb-v7g8t\" (UID: \"b30c08ad-dcd8-400b-8943-437f2d500b1a\") " pod="openshift-console/console-64d976c5fb-v7g8t" Sep 08 18:25:03.622754 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:03.622589 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b30c08ad-dcd8-400b-8943-437f2d500b1a-service-ca\") pod \"console-64d976c5fb-v7g8t\" (UID: \"b30c08ad-dcd8-400b-8943-437f2d500b1a\") " pod="openshift-console/console-64d976c5fb-v7g8t" Sep 08 18:25:03.622754 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:03.622622 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cz58h\" (UniqueName: \"kubernetes.io/projected/b30c08ad-dcd8-400b-8943-437f2d500b1a-kube-api-access-cz58h\") pod \"console-64d976c5fb-v7g8t\" (UID: \"b30c08ad-dcd8-400b-8943-437f2d500b1a\") " pod="openshift-console/console-64d976c5fb-v7g8t" Sep 08 18:25:03.622754 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:03.622720 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b30c08ad-dcd8-400b-8943-437f2d500b1a-console-oauth-config\") pod \"console-64d976c5fb-v7g8t\" (UID: \"b30c08ad-dcd8-400b-8943-437f2d500b1a\") " pod="openshift-console/console-64d976c5fb-v7g8t" Sep 08 18:25:03.622998 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:03.622766 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b30c08ad-dcd8-400b-8943-437f2d500b1a-trusted-ca-bundle\") pod \"console-64d976c5fb-v7g8t\" (UID: \"b30c08ad-dcd8-400b-8943-437f2d500b1a\") " pod="openshift-console/console-64d976c5fb-v7g8t" Sep 08 18:25:03.622998 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:03.622799 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b30c08ad-dcd8-400b-8943-437f2d500b1a-console-serving-cert\") pod \"console-64d976c5fb-v7g8t\" (UID: \"b30c08ad-dcd8-400b-8943-437f2d500b1a\") " pod="openshift-console/console-64d976c5fb-v7g8t" Sep 08 18:25:03.623359 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:03.623337 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b30c08ad-dcd8-400b-8943-437f2d500b1a-console-config\") pod \"console-64d976c5fb-v7g8t\" (UID: \"b30c08ad-dcd8-400b-8943-437f2d500b1a\") " pod="openshift-console/console-64d976c5fb-v7g8t" Sep 08 18:25:03.623465 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:03.623430 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b30c08ad-dcd8-400b-8943-437f2d500b1a-oauth-serving-cert\") pod \"console-64d976c5fb-v7g8t\" (UID: \"b30c08ad-dcd8-400b-8943-437f2d500b1a\") " pod="openshift-console/console-64d976c5fb-v7g8t" Sep 08 18:25:03.623465 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:03.623444 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b30c08ad-dcd8-400b-8943-437f2d500b1a-service-ca\") pod \"console-64d976c5fb-v7g8t\" (UID: \"b30c08ad-dcd8-400b-8943-437f2d500b1a\") " pod="openshift-console/console-64d976c5fb-v7g8t" Sep 08 18:25:03.623763 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:03.623739 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b30c08ad-dcd8-400b-8943-437f2d500b1a-trusted-ca-bundle\") pod \"console-64d976c5fb-v7g8t\" (UID: \"b30c08ad-dcd8-400b-8943-437f2d500b1a\") " pod="openshift-console/console-64d976c5fb-v7g8t" Sep 08 18:25:03.625555 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:03.625535 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b30c08ad-dcd8-400b-8943-437f2d500b1a-console-oauth-config\") pod \"console-64d976c5fb-v7g8t\" (UID: \"b30c08ad-dcd8-400b-8943-437f2d500b1a\") " pod="openshift-console/console-64d976c5fb-v7g8t" Sep 08 18:25:03.625848 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:03.625826 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b30c08ad-dcd8-400b-8943-437f2d500b1a-console-serving-cert\") pod \"console-64d976c5fb-v7g8t\" (UID: \"b30c08ad-dcd8-400b-8943-437f2d500b1a\") " pod="openshift-console/console-64d976c5fb-v7g8t" Sep 08 18:25:03.631417 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:03.631397 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cz58h\" (UniqueName: \"kubernetes.io/projected/b30c08ad-dcd8-400b-8943-437f2d500b1a-kube-api-access-cz58h\") pod \"console-64d976c5fb-v7g8t\" (UID: \"b30c08ad-dcd8-400b-8943-437f2d500b1a\") " pod="openshift-console/console-64d976c5fb-v7g8t" Sep 08 18:25:03.769343 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:03.769256 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64d976c5fb-v7g8t" Sep 08 18:25:04.078778 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:04.078734 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:04.180065 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:04.180008 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:04.877590 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:04.877536 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:04.983018 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:04.982977 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:05.474603 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:05.474570 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:05.574628 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:05.574595 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:06.653809 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:06.653773 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6c87577785-px7mf" Sep 08 18:25:06.935707 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:06.935588 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:07.043123 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:07.043089 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:07.735698 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:07.735641 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:07.835606 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:07.835576 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:08.214616 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:08.214580 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64d976c5fb-v7g8t"] Sep 08 18:25:08.587087 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:08.586926 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:08.687183 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:08.687115 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:08.759924 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:08.759889 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-ctmck" event={"ID":"b514b325-9daf-4c8f-a8c1-cee65d51c2f0","Type":"ContainerStarted","Data":"412505a8c23577fe61d8639aa9f3d08cd344ce752dce76c24a50f637df3c6e25"} Sep 08 18:25:08.760547 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:08.760526 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-ctmck" Sep 08 18:25:08.761698 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:08.761655 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64d976c5fb-v7g8t" event={"ID":"b30c08ad-dcd8-400b-8943-437f2d500b1a","Type":"ContainerStarted","Data":"586d136cb8dfc0437d6d5fd5effeb5ee3e01ea80a3f7b6ad522dc8f025ca4b25"} Sep 08 18:25:08.772893 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:08.772868 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-ctmck" Sep 08 18:25:08.784356 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:08.784301 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-ctmck" podStartSLOduration=4.260510733 podStartE2EDuration="21.784284268s" totalImagesPullingTime="17.523773535s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:24:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:25:08.78239406 +0000 UTC m=+90.988379459" watchObservedRunningTime="2026-09-08 18:25:08.784284268 +0000 UTC m=+90.990269791" Sep 08 18:25:08.791155 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:08.791087 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:08.913272 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:08.913169 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:08.990851 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:08.990810 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:09.472552 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:09.468848 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:09.570362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:09.569584 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:09.770639 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:09.770547 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-ggzxh" event={"ID":"1235e212-ee01-4797-8566-fc2b904316fd","Type":"ContainerStarted","Data":"ba4ae97d73f38b1537157bae39cbd5f995cb63a62109bffed7d16ad671b36de4"} Sep 08 18:25:09.770639 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:09.770610 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-ggzxh" event={"ID":"1235e212-ee01-4797-8566-fc2b904316fd","Type":"ContainerStarted","Data":"9b5599f4f27d16845429c1fcc14ccc50c0e83582fe127b367ad9ded7c44c5610"} Sep 08 18:25:09.801967 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:09.801885 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-ggzxh" podStartSLOduration=1.39298831 podStartE2EDuration="9.801865732s" totalImagesPullingTime="8.408877422s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:25:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:25:09.800560224 +0000 UTC m=+92.006545636" watchObservedRunningTime="2026-09-08 18:25:09.801865732 +0000 UTC m=+92.007851133" Sep 08 18:25:10.520396 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:10.520216 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6c87577785-px7mf"] Sep 08 18:25:10.610642 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:10.610595 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:10.710408 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:10.710373 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:10.811034 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:10.810996 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:11.234882 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.234731 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:11.457280 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.457217 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:11.479376 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.479341 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-dpnvv"] Sep 08 18:25:11.504719 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.504620 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-dpnvv"] Sep 08 18:25:11.505589 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.504880 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-dpnvv" Sep 08 18:25:11.508626 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.508589 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mdqr6"] Sep 08 18:25:11.510859 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.510374 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 08 18:25:11.510859 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.510406 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 08 18:25:11.510859 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.510663 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-srxgm\"" Sep 08 18:25:11.528573 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.528545 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mdqr6" Sep 08 18:25:11.532288 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.532074 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-f4lkk\"" Sep 08 18:25:11.532288 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.532174 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 08 18:25:11.532288 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.532241 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 08 18:25:11.534014 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.532916 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 08 18:25:11.554511 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.554480 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:11.605410 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.605374 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/06e53c2f-304e-403c-9ed9-d107bbcdf01c-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-dpnvv\" (UID: \"06e53c2f-304e-403c-9ed9-d107bbcdf01c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-dpnvv" Sep 08 18:25:11.605609 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.605468 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/06e53c2f-304e-403c-9ed9-d107bbcdf01c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-dpnvv\" (UID: \"06e53c2f-304e-403c-9ed9-d107bbcdf01c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-dpnvv" Sep 08 18:25:11.605609 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.605557 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/06e53c2f-304e-403c-9ed9-d107bbcdf01c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-dpnvv\" (UID: \"06e53c2f-304e-403c-9ed9-d107bbcdf01c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-dpnvv" Sep 08 18:25:11.605609 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.605593 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjg8z\" (UniqueName: \"kubernetes.io/projected/06e53c2f-304e-403c-9ed9-d107bbcdf01c-kube-api-access-sjg8z\") pod \"openshift-state-metrics-657fd6c54-dpnvv\" (UID: \"06e53c2f-304e-403c-9ed9-d107bbcdf01c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-dpnvv" Sep 08 18:25:11.659009 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.658941 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:11.706631 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.706591 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7e607d22-ab60-4fcc-88ad-ca0acfcf3642-node-exporter-textfile\") pod \"node-exporter-mdqr6\" (UID: \"7e607d22-ab60-4fcc-88ad-ca0acfcf3642\") " pod="openshift-monitoring/node-exporter-mdqr6" Sep 08 18:25:11.706850 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.706644 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7e607d22-ab60-4fcc-88ad-ca0acfcf3642-node-exporter-tls\") pod \"node-exporter-mdqr6\" (UID: \"7e607d22-ab60-4fcc-88ad-ca0acfcf3642\") " pod="openshift-monitoring/node-exporter-mdqr6" Sep 08 18:25:11.706850 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.706700 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e607d22-ab60-4fcc-88ad-ca0acfcf3642-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mdqr6\" (UID: \"7e607d22-ab60-4fcc-88ad-ca0acfcf3642\") " pod="openshift-monitoring/node-exporter-mdqr6" Sep 08 18:25:11.706850 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.706725 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7e607d22-ab60-4fcc-88ad-ca0acfcf3642-node-exporter-accelerators-collector-config\") pod \"node-exporter-mdqr6\" (UID: \"7e607d22-ab60-4fcc-88ad-ca0acfcf3642\") " pod="openshift-monitoring/node-exporter-mdqr6" Sep 08 18:25:11.707025 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.706850 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/06e53c2f-304e-403c-9ed9-d107bbcdf01c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-dpnvv\" (UID: \"06e53c2f-304e-403c-9ed9-d107bbcdf01c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-dpnvv" Sep 08 18:25:11.707025 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.706910 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgdvb\" (UniqueName: \"kubernetes.io/projected/7e607d22-ab60-4fcc-88ad-ca0acfcf3642-kube-api-access-zgdvb\") pod \"node-exporter-mdqr6\" (UID: \"7e607d22-ab60-4fcc-88ad-ca0acfcf3642\") " pod="openshift-monitoring/node-exporter-mdqr6" Sep 08 18:25:11.707025 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.706943 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7e607d22-ab60-4fcc-88ad-ca0acfcf3642-node-exporter-wtmp\") pod \"node-exporter-mdqr6\" (UID: \"7e607d22-ab60-4fcc-88ad-ca0acfcf3642\") " pod="openshift-monitoring/node-exporter-mdqr6" Sep 08 18:25:11.707025 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.706968 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e607d22-ab60-4fcc-88ad-ca0acfcf3642-metrics-client-ca\") pod \"node-exporter-mdqr6\" (UID: \"7e607d22-ab60-4fcc-88ad-ca0acfcf3642\") " pod="openshift-monitoring/node-exporter-mdqr6" Sep 08 18:25:11.707233 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.707070 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7e607d22-ab60-4fcc-88ad-ca0acfcf3642-root\") pod \"node-exporter-mdqr6\" (UID: \"7e607d22-ab60-4fcc-88ad-ca0acfcf3642\") " pod="openshift-monitoring/node-exporter-mdqr6" Sep 08 18:25:11.707233 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.707164 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/06e53c2f-304e-403c-9ed9-d107bbcdf01c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-dpnvv\" (UID: \"06e53c2f-304e-403c-9ed9-d107bbcdf01c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-dpnvv" Sep 08 18:25:11.707339 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.707283 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjg8z\" (UniqueName: \"kubernetes.io/projected/06e53c2f-304e-403c-9ed9-d107bbcdf01c-kube-api-access-sjg8z\") pod \"openshift-state-metrics-657fd6c54-dpnvv\" (UID: \"06e53c2f-304e-403c-9ed9-d107bbcdf01c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-dpnvv" Sep 08 18:25:11.707339 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.707325 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7e607d22-ab60-4fcc-88ad-ca0acfcf3642-sys\") pod \"node-exporter-mdqr6\" (UID: \"7e607d22-ab60-4fcc-88ad-ca0acfcf3642\") " pod="openshift-monitoring/node-exporter-mdqr6" Sep 08 18:25:11.707615 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.707587 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/06e53c2f-304e-403c-9ed9-d107bbcdf01c-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-dpnvv\" (UID: \"06e53c2f-304e-403c-9ed9-d107bbcdf01c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-dpnvv" Sep 08 18:25:11.708399 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.708372 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/06e53c2f-304e-403c-9ed9-d107bbcdf01c-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-dpnvv\" (UID: \"06e53c2f-304e-403c-9ed9-d107bbcdf01c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-dpnvv" Sep 08 18:25:11.710635 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.710581 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/06e53c2f-304e-403c-9ed9-d107bbcdf01c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-dpnvv\" (UID: \"06e53c2f-304e-403c-9ed9-d107bbcdf01c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-dpnvv" Sep 08 18:25:11.711073 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.711016 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/06e53c2f-304e-403c-9ed9-d107bbcdf01c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-dpnvv\" (UID: \"06e53c2f-304e-403c-9ed9-d107bbcdf01c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-dpnvv" Sep 08 18:25:11.716283 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.716256 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjg8z\" (UniqueName: \"kubernetes.io/projected/06e53c2f-304e-403c-9ed9-d107bbcdf01c-kube-api-access-sjg8z\") pod \"openshift-state-metrics-657fd6c54-dpnvv\" (UID: \"06e53c2f-304e-403c-9ed9-d107bbcdf01c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-dpnvv" Sep 08 18:25:11.808969 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.808932 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7e607d22-ab60-4fcc-88ad-ca0acfcf3642-node-exporter-textfile\") pod \"node-exporter-mdqr6\" (UID: \"7e607d22-ab60-4fcc-88ad-ca0acfcf3642\") " pod="openshift-monitoring/node-exporter-mdqr6" Sep 08 18:25:11.809163 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.808986 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7e607d22-ab60-4fcc-88ad-ca0acfcf3642-node-exporter-tls\") pod \"node-exporter-mdqr6\" (UID: \"7e607d22-ab60-4fcc-88ad-ca0acfcf3642\") " pod="openshift-monitoring/node-exporter-mdqr6" Sep 08 18:25:11.809278 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.809234 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e607d22-ab60-4fcc-88ad-ca0acfcf3642-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mdqr6\" (UID: \"7e607d22-ab60-4fcc-88ad-ca0acfcf3642\") " pod="openshift-monitoring/node-exporter-mdqr6" Sep 08 18:25:11.809461 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.809298 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7e607d22-ab60-4fcc-88ad-ca0acfcf3642-node-exporter-accelerators-collector-config\") pod \"node-exporter-mdqr6\" (UID: \"7e607d22-ab60-4fcc-88ad-ca0acfcf3642\") " pod="openshift-monitoring/node-exporter-mdqr6" Sep 08 18:25:11.809461 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.809367 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgdvb\" (UniqueName: \"kubernetes.io/projected/7e607d22-ab60-4fcc-88ad-ca0acfcf3642-kube-api-access-zgdvb\") pod \"node-exporter-mdqr6\" (UID: \"7e607d22-ab60-4fcc-88ad-ca0acfcf3642\") " pod="openshift-monitoring/node-exporter-mdqr6" Sep 08 18:25:11.810833 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.809397 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7e607d22-ab60-4fcc-88ad-ca0acfcf3642-node-exporter-wtmp\") pod \"node-exporter-mdqr6\" (UID: \"7e607d22-ab60-4fcc-88ad-ca0acfcf3642\") " pod="openshift-monitoring/node-exporter-mdqr6" Sep 08 18:25:11.810833 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.809615 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e607d22-ab60-4fcc-88ad-ca0acfcf3642-metrics-client-ca\") pod \"node-exporter-mdqr6\" (UID: \"7e607d22-ab60-4fcc-88ad-ca0acfcf3642\") " pod="openshift-monitoring/node-exporter-mdqr6" Sep 08 18:25:11.810833 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.809653 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7e607d22-ab60-4fcc-88ad-ca0acfcf3642-root\") pod \"node-exporter-mdqr6\" (UID: \"7e607d22-ab60-4fcc-88ad-ca0acfcf3642\") " pod="openshift-monitoring/node-exporter-mdqr6" Sep 08 18:25:11.810833 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.809716 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7e607d22-ab60-4fcc-88ad-ca0acfcf3642-sys\") pod \"node-exporter-mdqr6\" (UID: \"7e607d22-ab60-4fcc-88ad-ca0acfcf3642\") " pod="openshift-monitoring/node-exporter-mdqr6" Sep 08 18:25:11.810833 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.809805 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7e607d22-ab60-4fcc-88ad-ca0acfcf3642-sys\") pod \"node-exporter-mdqr6\" (UID: \"7e607d22-ab60-4fcc-88ad-ca0acfcf3642\") " pod="openshift-monitoring/node-exporter-mdqr6" Sep 08 18:25:11.810833 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.809842 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7e607d22-ab60-4fcc-88ad-ca0acfcf3642-root\") pod \"node-exporter-mdqr6\" (UID: \"7e607d22-ab60-4fcc-88ad-ca0acfcf3642\") " pod="openshift-monitoring/node-exporter-mdqr6" Sep 08 18:25:11.810833 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.809847 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7e607d22-ab60-4fcc-88ad-ca0acfcf3642-node-exporter-wtmp\") pod \"node-exporter-mdqr6\" (UID: \"7e607d22-ab60-4fcc-88ad-ca0acfcf3642\") " pod="openshift-monitoring/node-exporter-mdqr6" Sep 08 18:25:11.810833 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.809999 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7e607d22-ab60-4fcc-88ad-ca0acfcf3642-node-exporter-textfile\") pod \"node-exporter-mdqr6\" (UID: \"7e607d22-ab60-4fcc-88ad-ca0acfcf3642\") " pod="openshift-monitoring/node-exporter-mdqr6" Sep 08 18:25:11.810833 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.810351 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e607d22-ab60-4fcc-88ad-ca0acfcf3642-metrics-client-ca\") pod \"node-exporter-mdqr6\" (UID: \"7e607d22-ab60-4fcc-88ad-ca0acfcf3642\") " pod="openshift-monitoring/node-exporter-mdqr6" Sep 08 18:25:11.810833 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.810787 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7e607d22-ab60-4fcc-88ad-ca0acfcf3642-node-exporter-accelerators-collector-config\") pod \"node-exporter-mdqr6\" (UID: \"7e607d22-ab60-4fcc-88ad-ca0acfcf3642\") " pod="openshift-monitoring/node-exporter-mdqr6" Sep 08 18:25:11.812908 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.812884 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e607d22-ab60-4fcc-88ad-ca0acfcf3642-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mdqr6\" (UID: \"7e607d22-ab60-4fcc-88ad-ca0acfcf3642\") " pod="openshift-monitoring/node-exporter-mdqr6" Sep 08 18:25:11.814529 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.814502 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7e607d22-ab60-4fcc-88ad-ca0acfcf3642-node-exporter-tls\") pod \"node-exporter-mdqr6\" (UID: \"7e607d22-ab60-4fcc-88ad-ca0acfcf3642\") " pod="openshift-monitoring/node-exporter-mdqr6" Sep 08 18:25:11.819917 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.819626 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-dpnvv" Sep 08 18:25:11.820103 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.820052 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgdvb\" (UniqueName: \"kubernetes.io/projected/7e607d22-ab60-4fcc-88ad-ca0acfcf3642-kube-api-access-zgdvb\") pod \"node-exporter-mdqr6\" (UID: \"7e607d22-ab60-4fcc-88ad-ca0acfcf3642\") " pod="openshift-monitoring/node-exporter-mdqr6" Sep 08 18:25:11.842200 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.842141 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mdqr6" Sep 08 18:25:11.923833 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:11.923792 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:12.086149 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:12.086072 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:12.353668 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:25:12.353636 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e607d22_ab60_4fcc_88ad_ca0acfcf3642.slice/crio-f9fa0c380339750a9226c23d41e2e868658cc2cfeac3f972a15eb82e3ce6c8c0 WatchSource:0}: Error finding container f9fa0c380339750a9226c23d41e2e868658cc2cfeac3f972a15eb82e3ce6c8c0: Status 404 returned error can't find the container with id f9fa0c380339750a9226c23d41e2e868658cc2cfeac3f972a15eb82e3ce6c8c0 Sep 08 18:25:12.498575 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:12.498535 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:12.612395 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:12.612324 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:12.633589 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:12.633551 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-dpnvv"] Sep 08 18:25:12.700233 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:12.700194 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:12.789373 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:12.789339 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mdqr6" event={"ID":"7e607d22-ab60-4fcc-88ad-ca0acfcf3642","Type":"ContainerStarted","Data":"f9fa0c380339750a9226c23d41e2e868658cc2cfeac3f972a15eb82e3ce6c8c0"} Sep 08 18:25:12.791075 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:12.791047 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-dpnvv" event={"ID":"06e53c2f-304e-403c-9ed9-d107bbcdf01c","Type":"ContainerStarted","Data":"a8d1d7dc0952209df2ca4c1fae50405fba6e018546a7f0b1bb09e22f56ac6546"} Sep 08 18:25:13.798022 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:13.797982 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-dpnvv" event={"ID":"06e53c2f-304e-403c-9ed9-d107bbcdf01c","Type":"ContainerStarted","Data":"b7efc145087dba7cf58765b09dd730a0cc809f7be138be073f3102a1b8536cfb"} Sep 08 18:25:13.798544 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:13.798059 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-dpnvv" event={"ID":"06e53c2f-304e-403c-9ed9-d107bbcdf01c","Type":"ContainerStarted","Data":"0252dfeefa2fc4695d1cc75f30f0b408f7b3e3e4ed904cec60c5a7f77479e69c"} Sep 08 18:25:13.799893 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:13.799862 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64d976c5fb-v7g8t" event={"ID":"b30c08ad-dcd8-400b-8943-437f2d500b1a","Type":"ContainerStarted","Data":"e8299189b8ff484b89b459ade4cff936b4f3b79876e11dd2c20abffcc043bdfd"} Sep 08 18:25:13.830358 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:13.830287 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-64d976c5fb-v7g8t" podStartSLOduration=6.329862383 podStartE2EDuration="10.830265955s" totalImagesPullingTime="4.500403572s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:25:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:25:13.828916298 +0000 UTC m=+96.034901698" watchObservedRunningTime="2026-09-08 18:25:13.830265955 +0000 UTC m=+96.036251356" Sep 08 18:25:14.470239 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.470142 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:14.490556 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.490519 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-547cd64fc8-vx522"] Sep 08 18:25:14.513790 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.513559 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-547cd64fc8-vx522"] Sep 08 18:25:14.513790 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.513751 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-547cd64fc8-vx522" Sep 08 18:25:14.516434 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.516407 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Sep 08 18:25:14.517255 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.516666 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Sep 08 18:25:14.519901 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.516895 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-jqlqx\"" Sep 08 18:25:14.519901 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.516988 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Sep 08 18:25:14.519901 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.517050 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Sep 08 18:25:14.519901 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.517161 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Sep 08 18:25:14.519901 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.519722 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-bqrs0tnjp063n\"" Sep 08 18:25:14.539107 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.539081 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:14.634705 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.634659 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:14.636737 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.636705 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c5505916-4939-4ed0-8168-a469bcb150a9-secret-thanos-querier-tls\") pod \"thanos-querier-547cd64fc8-vx522\" (UID: \"c5505916-4939-4ed0-8168-a469bcb150a9\") " pod="openshift-monitoring/thanos-querier-547cd64fc8-vx522" Sep 08 18:25:14.636892 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.636768 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c5505916-4939-4ed0-8168-a469bcb150a9-metrics-client-ca\") pod \"thanos-querier-547cd64fc8-vx522\" (UID: \"c5505916-4939-4ed0-8168-a469bcb150a9\") " pod="openshift-monitoring/thanos-querier-547cd64fc8-vx522" Sep 08 18:25:14.636892 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.636791 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tvtf\" (UniqueName: \"kubernetes.io/projected/c5505916-4939-4ed0-8168-a469bcb150a9-kube-api-access-2tvtf\") pod \"thanos-querier-547cd64fc8-vx522\" (UID: \"c5505916-4939-4ed0-8168-a469bcb150a9\") " pod="openshift-monitoring/thanos-querier-547cd64fc8-vx522" Sep 08 18:25:14.636892 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.636814 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c5505916-4939-4ed0-8168-a469bcb150a9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-547cd64fc8-vx522\" (UID: \"c5505916-4939-4ed0-8168-a469bcb150a9\") " pod="openshift-monitoring/thanos-querier-547cd64fc8-vx522" Sep 08 18:25:14.636892 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.636848 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c5505916-4939-4ed0-8168-a469bcb150a9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-547cd64fc8-vx522\" (UID: \"c5505916-4939-4ed0-8168-a469bcb150a9\") " pod="openshift-monitoring/thanos-querier-547cd64fc8-vx522" Sep 08 18:25:14.636892 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.636874 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c5505916-4939-4ed0-8168-a469bcb150a9-secret-grpc-tls\") pod \"thanos-querier-547cd64fc8-vx522\" (UID: \"c5505916-4939-4ed0-8168-a469bcb150a9\") " pod="openshift-monitoring/thanos-querier-547cd64fc8-vx522" Sep 08 18:25:14.637153 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.636939 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c5505916-4939-4ed0-8168-a469bcb150a9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-547cd64fc8-vx522\" (UID: \"c5505916-4939-4ed0-8168-a469bcb150a9\") " pod="openshift-monitoring/thanos-querier-547cd64fc8-vx522" Sep 08 18:25:14.637153 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.636966 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c5505916-4939-4ed0-8168-a469bcb150a9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-547cd64fc8-vx522\" (UID: \"c5505916-4939-4ed0-8168-a469bcb150a9\") " pod="openshift-monitoring/thanos-querier-547cd64fc8-vx522" Sep 08 18:25:14.734970 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.734887 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:14.738176 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.738149 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c5505916-4939-4ed0-8168-a469bcb150a9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-547cd64fc8-vx522\" (UID: \"c5505916-4939-4ed0-8168-a469bcb150a9\") " pod="openshift-monitoring/thanos-querier-547cd64fc8-vx522" Sep 08 18:25:14.738176 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.738180 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c5505916-4939-4ed0-8168-a469bcb150a9-secret-grpc-tls\") pod \"thanos-querier-547cd64fc8-vx522\" (UID: \"c5505916-4939-4ed0-8168-a469bcb150a9\") " pod="openshift-monitoring/thanos-querier-547cd64fc8-vx522" Sep 08 18:25:14.738369 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.738218 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c5505916-4939-4ed0-8168-a469bcb150a9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-547cd64fc8-vx522\" (UID: \"c5505916-4939-4ed0-8168-a469bcb150a9\") " pod="openshift-monitoring/thanos-querier-547cd64fc8-vx522" Sep 08 18:25:14.738369 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.738248 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c5505916-4939-4ed0-8168-a469bcb150a9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-547cd64fc8-vx522\" (UID: \"c5505916-4939-4ed0-8168-a469bcb150a9\") " pod="openshift-monitoring/thanos-querier-547cd64fc8-vx522" Sep 08 18:25:14.738369 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.738313 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c5505916-4939-4ed0-8168-a469bcb150a9-secret-thanos-querier-tls\") pod \"thanos-querier-547cd64fc8-vx522\" (UID: \"c5505916-4939-4ed0-8168-a469bcb150a9\") " pod="openshift-monitoring/thanos-querier-547cd64fc8-vx522" Sep 08 18:25:14.738369 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.738358 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c5505916-4939-4ed0-8168-a469bcb150a9-metrics-client-ca\") pod \"thanos-querier-547cd64fc8-vx522\" (UID: \"c5505916-4939-4ed0-8168-a469bcb150a9\") " pod="openshift-monitoring/thanos-querier-547cd64fc8-vx522" Sep 08 18:25:14.738561 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.738381 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tvtf\" (UniqueName: \"kubernetes.io/projected/c5505916-4939-4ed0-8168-a469bcb150a9-kube-api-access-2tvtf\") pod \"thanos-querier-547cd64fc8-vx522\" (UID: \"c5505916-4939-4ed0-8168-a469bcb150a9\") " pod="openshift-monitoring/thanos-querier-547cd64fc8-vx522" Sep 08 18:25:14.738561 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.738407 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c5505916-4939-4ed0-8168-a469bcb150a9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-547cd64fc8-vx522\" (UID: \"c5505916-4939-4ed0-8168-a469bcb150a9\") " pod="openshift-monitoring/thanos-querier-547cd64fc8-vx522" Sep 08 18:25:14.739603 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.739554 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c5505916-4939-4ed0-8168-a469bcb150a9-metrics-client-ca\") pod \"thanos-querier-547cd64fc8-vx522\" (UID: \"c5505916-4939-4ed0-8168-a469bcb150a9\") " pod="openshift-monitoring/thanos-querier-547cd64fc8-vx522" Sep 08 18:25:14.741361 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.741339 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c5505916-4939-4ed0-8168-a469bcb150a9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-547cd64fc8-vx522\" (UID: \"c5505916-4939-4ed0-8168-a469bcb150a9\") " pod="openshift-monitoring/thanos-querier-547cd64fc8-vx522" Sep 08 18:25:14.741495 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.741474 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c5505916-4939-4ed0-8168-a469bcb150a9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-547cd64fc8-vx522\" (UID: \"c5505916-4939-4ed0-8168-a469bcb150a9\") " pod="openshift-monitoring/thanos-querier-547cd64fc8-vx522" Sep 08 18:25:14.741589 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.741507 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c5505916-4939-4ed0-8168-a469bcb150a9-secret-grpc-tls\") pod \"thanos-querier-547cd64fc8-vx522\" (UID: \"c5505916-4939-4ed0-8168-a469bcb150a9\") " pod="openshift-monitoring/thanos-querier-547cd64fc8-vx522" Sep 08 18:25:14.743062 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.743039 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c5505916-4939-4ed0-8168-a469bcb150a9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-547cd64fc8-vx522\" (UID: \"c5505916-4939-4ed0-8168-a469bcb150a9\") " pod="openshift-monitoring/thanos-querier-547cd64fc8-vx522" Sep 08 18:25:14.747637 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.743861 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c5505916-4939-4ed0-8168-a469bcb150a9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-547cd64fc8-vx522\" (UID: \"c5505916-4939-4ed0-8168-a469bcb150a9\") " pod="openshift-monitoring/thanos-querier-547cd64fc8-vx522" Sep 08 18:25:14.747637 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.747236 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c5505916-4939-4ed0-8168-a469bcb150a9-secret-thanos-querier-tls\") pod \"thanos-querier-547cd64fc8-vx522\" (UID: \"c5505916-4939-4ed0-8168-a469bcb150a9\") " pod="openshift-monitoring/thanos-querier-547cd64fc8-vx522" Sep 08 18:25:14.751055 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.751009 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tvtf\" (UniqueName: \"kubernetes.io/projected/c5505916-4939-4ed0-8168-a469bcb150a9-kube-api-access-2tvtf\") pod \"thanos-querier-547cd64fc8-vx522\" (UID: \"c5505916-4939-4ed0-8168-a469bcb150a9\") " pod="openshift-monitoring/thanos-querier-547cd64fc8-vx522" Sep 08 18:25:14.806997 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.806957 2555 generic.go:356] "Generic (PLEG): container finished" podID="7e607d22-ab60-4fcc-88ad-ca0acfcf3642" containerID="4dfc9fb1710262343d3e1cba707bf5269f02ad1d1b89282d84d4d9734c49d6c2" exitCode=0 Sep 08 18:25:14.807467 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.807053 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mdqr6" event={"ID":"7e607d22-ab60-4fcc-88ad-ca0acfcf3642","Type":"ContainerDied","Data":"4dfc9fb1710262343d3e1cba707bf5269f02ad1d1b89282d84d4d9734c49d6c2"} Sep 08 18:25:14.828062 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:14.828045 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-547cd64fc8-vx522" Sep 08 18:25:15.383126 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:15.383011 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-547cd64fc8-vx522"] Sep 08 18:25:15.386175 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:25:15.386147 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5505916_4939_4ed0_8168_a469bcb150a9.slice/crio-bf7544b736179a6d7e3eb492f705e59c2bc8615693dedfd06f03ac6086de0038 WatchSource:0}: Error finding container bf7544b736179a6d7e3eb492f705e59c2bc8615693dedfd06f03ac6086de0038: Status 404 returned error can't find the container with id bf7544b736179a6d7e3eb492f705e59c2bc8615693dedfd06f03ac6086de0038 Sep 08 18:25:15.814282 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:15.814244 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mdqr6" event={"ID":"7e607d22-ab60-4fcc-88ad-ca0acfcf3642","Type":"ContainerStarted","Data":"4cfddb010b603cacc06b2f66bae7f7eeb34bf1aa0b65354d3b36acda3917bad8"} Sep 08 18:25:15.814778 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:15.814317 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mdqr6" event={"ID":"7e607d22-ab60-4fcc-88ad-ca0acfcf3642","Type":"ContainerStarted","Data":"81f8c6cc8e8f8b0dda6d3ae6c8a8e6bca5ee185a0800885c9ddc693bb6619ce0"} Sep 08 18:25:15.817240 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:15.817211 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-dpnvv" event={"ID":"06e53c2f-304e-403c-9ed9-d107bbcdf01c","Type":"ContainerStarted","Data":"86c0ff47292fc48af56960d5a05fc01fe0776f7be818d8a5a64dafded536be28"} Sep 08 18:25:15.819075 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:15.819051 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-547cd64fc8-vx522" event={"ID":"c5505916-4939-4ed0-8168-a469bcb150a9","Type":"ContainerStarted","Data":"bf7544b736179a6d7e3eb492f705e59c2bc8615693dedfd06f03ac6086de0038"} Sep 08 18:25:15.835361 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:15.835318 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mdqr6" podStartSLOduration=3.087267948 podStartE2EDuration="4.835302033s" totalImagesPullingTime="1.650666447s" totalInitContainerRuntime="97.367638ms" isStatefulPod=true podCreationTimestamp="2026-09-08 18:25:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:25:15.833217792 +0000 UTC m=+98.039203201" watchObservedRunningTime="2026-09-08 18:25:15.835302033 +0000 UTC m=+98.041287473" Sep 08 18:25:15.850843 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:15.850793 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-dpnvv" podStartSLOduration=2.723056085 podStartE2EDuration="4.850775183s" totalImagesPullingTime="2.127719098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:25:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:25:15.849496847 +0000 UTC m=+98.055482246" watchObservedRunningTime="2026-09-08 18:25:15.850775183 +0000 UTC m=+98.056760581" Sep 08 18:25:16.259529 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:16.257580 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-ctvlx"] Sep 08 18:25:16.260211 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:16.260134 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:16.279350 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:16.277779 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-ctvlx"] Sep 08 18:25:16.279350 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:16.277927 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-ctvlx" Sep 08 18:25:16.282734 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:16.282708 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-lwxlx\"" Sep 08 18:25:16.283040 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:16.282918 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Sep 08 18:25:16.285215 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:16.284547 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64d976c5fb-v7g8t"] Sep 08 18:25:16.354089 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:16.353936 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:16.412000 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:16.411961 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:16.454828 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:16.454629 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b91f2cbc-e834-418d-a40b-e2387fac9385-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-ctvlx\" (UID: \"b91f2cbc-e834-418d-a40b-e2387fac9385\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-ctvlx" Sep 08 18:25:16.454828 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:16.454731 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-426sg\" (UniqueName: \"kubernetes.io/projected/b91f2cbc-e834-418d-a40b-e2387fac9385-kube-api-access-426sg\") pod \"monitoring-plugin-85bbbbbf4-ctvlx\" (UID: \"b91f2cbc-e834-418d-a40b-e2387fac9385\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-ctvlx" Sep 08 18:25:16.556077 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:16.556039 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b91f2cbc-e834-418d-a40b-e2387fac9385-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-ctvlx\" (UID: \"b91f2cbc-e834-418d-a40b-e2387fac9385\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-ctvlx" Sep 08 18:25:16.556258 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:16.556120 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-426sg\" (UniqueName: \"kubernetes.io/projected/b91f2cbc-e834-418d-a40b-e2387fac9385-kube-api-access-426sg\") pod \"monitoring-plugin-85bbbbbf4-ctvlx\" (UID: \"b91f2cbc-e834-418d-a40b-e2387fac9385\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-ctvlx" Sep 08 18:25:16.559039 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:16.559010 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b91f2cbc-e834-418d-a40b-e2387fac9385-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-ctvlx\" (UID: \"b91f2cbc-e834-418d-a40b-e2387fac9385\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-ctvlx" Sep 08 18:25:16.564475 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:16.564447 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-426sg\" (UniqueName: \"kubernetes.io/projected/b91f2cbc-e834-418d-a40b-e2387fac9385-kube-api-access-426sg\") pod \"monitoring-plugin-85bbbbbf4-ctvlx\" (UID: \"b91f2cbc-e834-418d-a40b-e2387fac9385\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-ctvlx" Sep 08 18:25:16.602755 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:16.602722 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-ctvlx" Sep 08 18:25:16.905573 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:16.905474 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-ctvlx"] Sep 08 18:25:16.908814 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:25:16.908789 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb91f2cbc_e834_418d_a40b_e2387fac9385.slice/crio-8494ba431be933b98c167b66c7236b17fb616404e0d1c675764b497e9d3879c8 WatchSource:0}: Error finding container 8494ba431be933b98c167b66c7236b17fb616404e0d1c675764b497e9d3879c8: Status 404 returned error can't find the container with id 8494ba431be933b98c167b66c7236b17fb616404e0d1c675764b497e9d3879c8 Sep 08 18:25:17.617635 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:17.617591 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:17.718072 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:17.718036 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:17.811954 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:17.811920 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:17.834497 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:17.834456 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-ctvlx" event={"ID":"b91f2cbc-e834-418d-a40b-e2387fac9385","Type":"ContainerStarted","Data":"8494ba431be933b98c167b66c7236b17fb616404e0d1c675764b497e9d3879c8"} Sep 08 18:25:19.844969 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:19.844924 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-ctvlx" event={"ID":"b91f2cbc-e834-418d-a40b-e2387fac9385","Type":"ContainerStarted","Data":"3f254d775884f57abe77ff9c23f0cff64fce59a5f8af447e611870781c6f91c9"} Sep 08 18:25:19.845448 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:19.844999 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-ctvlx" Sep 08 18:25:19.847375 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:19.847341 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-547cd64fc8-vx522" event={"ID":"c5505916-4939-4ed0-8168-a469bcb150a9","Type":"ContainerStarted","Data":"efc3d623f03be4df29e2787f3ea4da0a6bcd659b04114d997db5b3042a8999c8"} Sep 08 18:25:19.847489 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:19.847395 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-547cd64fc8-vx522" event={"ID":"c5505916-4939-4ed0-8168-a469bcb150a9","Type":"ContainerStarted","Data":"9c8dc6efce0b422f183deee0a729b41a6be7adb0974da5985052507cad96b745"} Sep 08 18:25:19.847489 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:19.847420 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-547cd64fc8-vx522" event={"ID":"c5505916-4939-4ed0-8168-a469bcb150a9","Type":"ContainerStarted","Data":"9ac7178039b8daa30f69ee8c5d0467ac2fda29d68df9098b2d89a660675c168e"} Sep 08 18:25:19.850217 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:19.850196 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-ctvlx" Sep 08 18:25:19.860675 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:19.860630 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-ctvlx" podStartSLOduration=1.5135688360000001 podStartE2EDuration="3.86061626s" totalImagesPullingTime="2.347047424s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:25:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:25:19.860294321 +0000 UTC m=+102.066279722" watchObservedRunningTime="2026-09-08 18:25:19.86061626 +0000 UTC m=+102.066601657" Sep 08 18:25:20.853219 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:20.853191 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-547cd64fc8-vx522" event={"ID":"c5505916-4939-4ed0-8168-a469bcb150a9","Type":"ContainerStarted","Data":"9cfe8b24791f23d3b2b98ec156646f1a0369429e3aa46f0606ce74625aaa6c73"} Sep 08 18:25:21.678700 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:21.678654 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-tqthm" Sep 08 18:25:21.859995 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:21.859954 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-547cd64fc8-vx522" event={"ID":"c5505916-4939-4ed0-8168-a469bcb150a9","Type":"ContainerStarted","Data":"f4593136380f831bbbdf234c053e2a5fbd487c656528bb175f9990d43ff743ee"} Sep 08 18:25:21.860454 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:21.860026 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-547cd64fc8-vx522" event={"ID":"c5505916-4939-4ed0-8168-a469bcb150a9","Type":"ContainerStarted","Data":"c9521769e931e000cdceb3870f852cee517c901526b84b2ef68b87d0b25e868a"} Sep 08 18:25:21.883786 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:21.883736 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-547cd64fc8-vx522" podStartSLOduration=2.888273465 podStartE2EDuration="7.883714828s" totalImagesPullingTime="4.995441363s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:25:14 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:25:21.882116007 +0000 UTC m=+104.088101419" watchObservedRunningTime="2026-09-08 18:25:21.883714828 +0000 UTC m=+104.089700226" Sep 08 18:25:22.132540 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:22.132501 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:22.196619 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:22.196568 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:22.864089 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:22.864051 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-547cd64fc8-vx522" Sep 08 18:25:23.770092 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:23.770058 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-64d976c5fb-v7g8t" Sep 08 18:25:26.833083 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:26.833049 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:26.887355 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:26.887305 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:26.940594 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:26.940557 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:28.873915 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:28.873886 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-547cd64fc8-vx522" Sep 08 18:25:35.548299 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:35.548239 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6c87577785-px7mf" podUID="612d5222-58c3-4da0-adc4-760f3fcd1843" containerName="registry" containerID="cri-o://af065103851dab2814d45c86c5c04d6793f8ab5ab02346eb3bcf00e961fc0fd8" gracePeriod=30 Sep 08 18:25:35.788478 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:35.788454 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c87577785-px7mf" Sep 08 18:25:35.819437 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:35.819364 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/612d5222-58c3-4da0-adc4-760f3fcd1843-installation-pull-secrets\") pod \"612d5222-58c3-4da0-adc4-760f3fcd1843\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " Sep 08 18:25:35.819437 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:35.819397 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/612d5222-58c3-4da0-adc4-760f3fcd1843-trusted-ca\") pod \"612d5222-58c3-4da0-adc4-760f3fcd1843\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " Sep 08 18:25:35.819626 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:35.819443 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/612d5222-58c3-4da0-adc4-760f3fcd1843-image-registry-private-configuration\") pod \"612d5222-58c3-4da0-adc4-760f3fcd1843\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " Sep 08 18:25:35.819626 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:35.819461 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/612d5222-58c3-4da0-adc4-760f3fcd1843-bound-sa-token\") pod \"612d5222-58c3-4da0-adc4-760f3fcd1843\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " Sep 08 18:25:35.819626 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:35.819486 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/612d5222-58c3-4da0-adc4-760f3fcd1843-registry-certificates\") pod \"612d5222-58c3-4da0-adc4-760f3fcd1843\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " Sep 08 18:25:35.819626 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:35.819516 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/612d5222-58c3-4da0-adc4-760f3fcd1843-ca-trust-extracted\") pod \"612d5222-58c3-4da0-adc4-760f3fcd1843\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " Sep 08 18:25:35.819626 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:35.819549 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-78rp2\" (UniqueName: \"kubernetes.io/projected/612d5222-58c3-4da0-adc4-760f3fcd1843-kube-api-access-78rp2\") pod \"612d5222-58c3-4da0-adc4-760f3fcd1843\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " Sep 08 18:25:35.819626 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:35.819588 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/612d5222-58c3-4da0-adc4-760f3fcd1843-registry-tls\") pod \"612d5222-58c3-4da0-adc4-760f3fcd1843\" (UID: \"612d5222-58c3-4da0-adc4-760f3fcd1843\") " Sep 08 18:25:35.820085 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:35.820021 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/612d5222-58c3-4da0-adc4-760f3fcd1843-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "612d5222-58c3-4da0-adc4-760f3fcd1843" (UID: "612d5222-58c3-4da0-adc4-760f3fcd1843"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:25:35.820085 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:35.820031 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/612d5222-58c3-4da0-adc4-760f3fcd1843-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "612d5222-58c3-4da0-adc4-760f3fcd1843" (UID: "612d5222-58c3-4da0-adc4-760f3fcd1843"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:25:35.822249 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:35.822217 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/612d5222-58c3-4da0-adc4-760f3fcd1843-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "612d5222-58c3-4da0-adc4-760f3fcd1843" (UID: "612d5222-58c3-4da0-adc4-760f3fcd1843"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:25:35.822405 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:35.822383 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/612d5222-58c3-4da0-adc4-760f3fcd1843-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "612d5222-58c3-4da0-adc4-760f3fcd1843" (UID: "612d5222-58c3-4da0-adc4-760f3fcd1843"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 18:25:35.822551 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:35.822534 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/612d5222-58c3-4da0-adc4-760f3fcd1843-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "612d5222-58c3-4da0-adc4-760f3fcd1843" (UID: "612d5222-58c3-4da0-adc4-760f3fcd1843"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:25:35.822633 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:35.822607 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/612d5222-58c3-4da0-adc4-760f3fcd1843-kube-api-access-78rp2" (OuterVolumeSpecName: "kube-api-access-78rp2") pod "612d5222-58c3-4da0-adc4-760f3fcd1843" (UID: "612d5222-58c3-4da0-adc4-760f3fcd1843"). InnerVolumeSpecName "kube-api-access-78rp2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:25:35.822759 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:35.822707 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/612d5222-58c3-4da0-adc4-760f3fcd1843-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "612d5222-58c3-4da0-adc4-760f3fcd1843" (UID: "612d5222-58c3-4da0-adc4-760f3fcd1843"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 18:25:35.829058 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:35.829027 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/612d5222-58c3-4da0-adc4-760f3fcd1843-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "612d5222-58c3-4da0-adc4-760f3fcd1843" (UID: "612d5222-58c3-4da0-adc4-760f3fcd1843"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:25:35.916478 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:35.916443 2555 generic.go:356] "Generic (PLEG): container finished" podID="612d5222-58c3-4da0-adc4-760f3fcd1843" containerID="af065103851dab2814d45c86c5c04d6793f8ab5ab02346eb3bcf00e961fc0fd8" exitCode=0 Sep 08 18:25:35.916638 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:35.916489 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c87577785-px7mf" Sep 08 18:25:35.916638 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:35.916512 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c87577785-px7mf" event={"ID":"612d5222-58c3-4da0-adc4-760f3fcd1843","Type":"ContainerDied","Data":"af065103851dab2814d45c86c5c04d6793f8ab5ab02346eb3bcf00e961fc0fd8"} Sep 08 18:25:35.916638 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:35.916551 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c87577785-px7mf" event={"ID":"612d5222-58c3-4da0-adc4-760f3fcd1843","Type":"ContainerDied","Data":"13fd40dc7dccad0e3eeec99a8d668c2ffdedb26a427124940c1594d382017f11"} Sep 08 18:25:35.916638 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:35.916570 2555 scope.go:117] "RemoveContainer" containerID="af065103851dab2814d45c86c5c04d6793f8ab5ab02346eb3bcf00e961fc0fd8" Sep 08 18:25:35.920247 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:35.920221 2555 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/612d5222-58c3-4da0-adc4-760f3fcd1843-registry-tls\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:25:35.920357 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:35.920253 2555 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/612d5222-58c3-4da0-adc4-760f3fcd1843-installation-pull-secrets\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:25:35.920357 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:35.920269 2555 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/612d5222-58c3-4da0-adc4-760f3fcd1843-trusted-ca\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:25:35.920357 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:35.920280 2555 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/612d5222-58c3-4da0-adc4-760f3fcd1843-image-registry-private-configuration\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:25:35.920357 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:35.920289 2555 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/612d5222-58c3-4da0-adc4-760f3fcd1843-bound-sa-token\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:25:35.920357 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:35.920298 2555 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/612d5222-58c3-4da0-adc4-760f3fcd1843-registry-certificates\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:25:35.920357 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:35.920305 2555 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/612d5222-58c3-4da0-adc4-760f3fcd1843-ca-trust-extracted\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:25:35.920357 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:35.920313 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78rp2\" (UniqueName: \"kubernetes.io/projected/612d5222-58c3-4da0-adc4-760f3fcd1843-kube-api-access-78rp2\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:25:35.925729 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:35.925670 2555 scope.go:117] "RemoveContainer" containerID="af065103851dab2814d45c86c5c04d6793f8ab5ab02346eb3bcf00e961fc0fd8" Sep 08 18:25:35.925992 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:25:35.925969 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af065103851dab2814d45c86c5c04d6793f8ab5ab02346eb3bcf00e961fc0fd8\": container with ID starting with af065103851dab2814d45c86c5c04d6793f8ab5ab02346eb3bcf00e961fc0fd8 not found: ID does not exist" containerID="af065103851dab2814d45c86c5c04d6793f8ab5ab02346eb3bcf00e961fc0fd8" Sep 08 18:25:35.926039 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:35.926000 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af065103851dab2814d45c86c5c04d6793f8ab5ab02346eb3bcf00e961fc0fd8"} err="failed to get container status \"af065103851dab2814d45c86c5c04d6793f8ab5ab02346eb3bcf00e961fc0fd8\": rpc error: code = NotFound desc = could not find container \"af065103851dab2814d45c86c5c04d6793f8ab5ab02346eb3bcf00e961fc0fd8\": container with ID starting with af065103851dab2814d45c86c5c04d6793f8ab5ab02346eb3bcf00e961fc0fd8 not found: ID does not exist" Sep 08 18:25:35.935979 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:35.935957 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6c87577785-px7mf"] Sep 08 18:25:35.941369 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:35.941343 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6c87577785-px7mf"] Sep 08 18:25:36.323249 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:36.323216 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="612d5222-58c3-4da0-adc4-760f3fcd1843" path="/var/lib/kubelet/pods/612d5222-58c3-4da0-adc4-760f3fcd1843/volumes" Sep 08 18:25:37.114819 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:37.114784 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:37.191054 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:37.191003 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:41.327263 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:41.327203 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-64d976c5fb-v7g8t" podUID="b30c08ad-dcd8-400b-8943-437f2d500b1a" containerName="console" containerID="cri-o://e8299189b8ff484b89b459ade4cff936b4f3b79876e11dd2c20abffcc043bdfd" gracePeriod=15 Sep 08 18:25:41.617045 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:41.617007 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64d976c5fb-v7g8t" Sep 08 18:25:41.661213 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:41.661186 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b30c08ad-dcd8-400b-8943-437f2d500b1a-trusted-ca-bundle\") pod \"b30c08ad-dcd8-400b-8943-437f2d500b1a\" (UID: \"b30c08ad-dcd8-400b-8943-437f2d500b1a\") " Sep 08 18:25:41.661383 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:41.661247 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b30c08ad-dcd8-400b-8943-437f2d500b1a-console-config\") pod \"b30c08ad-dcd8-400b-8943-437f2d500b1a\" (UID: \"b30c08ad-dcd8-400b-8943-437f2d500b1a\") " Sep 08 18:25:41.661383 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:41.661279 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b30c08ad-dcd8-400b-8943-437f2d500b1a-console-serving-cert\") pod \"b30c08ad-dcd8-400b-8943-437f2d500b1a\" (UID: \"b30c08ad-dcd8-400b-8943-437f2d500b1a\") " Sep 08 18:25:41.661383 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:41.661302 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cz58h\" (UniqueName: \"kubernetes.io/projected/b30c08ad-dcd8-400b-8943-437f2d500b1a-kube-api-access-cz58h\") pod \"b30c08ad-dcd8-400b-8943-437f2d500b1a\" (UID: \"b30c08ad-dcd8-400b-8943-437f2d500b1a\") " Sep 08 18:25:41.661383 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:41.661318 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b30c08ad-dcd8-400b-8943-437f2d500b1a-oauth-serving-cert\") pod \"b30c08ad-dcd8-400b-8943-437f2d500b1a\" (UID: \"b30c08ad-dcd8-400b-8943-437f2d500b1a\") " Sep 08 18:25:41.661383 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:41.661370 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b30c08ad-dcd8-400b-8943-437f2d500b1a-service-ca\") pod \"b30c08ad-dcd8-400b-8943-437f2d500b1a\" (UID: \"b30c08ad-dcd8-400b-8943-437f2d500b1a\") " Sep 08 18:25:41.661657 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:41.661397 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b30c08ad-dcd8-400b-8943-437f2d500b1a-console-oauth-config\") pod \"b30c08ad-dcd8-400b-8943-437f2d500b1a\" (UID: \"b30c08ad-dcd8-400b-8943-437f2d500b1a\") " Sep 08 18:25:41.661743 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:41.661599 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b30c08ad-dcd8-400b-8943-437f2d500b1a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b30c08ad-dcd8-400b-8943-437f2d500b1a" (UID: "b30c08ad-dcd8-400b-8943-437f2d500b1a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:25:41.661808 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:41.661780 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b30c08ad-dcd8-400b-8943-437f2d500b1a-service-ca" (OuterVolumeSpecName: "service-ca") pod "b30c08ad-dcd8-400b-8943-437f2d500b1a" (UID: "b30c08ad-dcd8-400b-8943-437f2d500b1a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:25:41.661861 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:41.661801 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b30c08ad-dcd8-400b-8943-437f2d500b1a-console-config" (OuterVolumeSpecName: "console-config") pod "b30c08ad-dcd8-400b-8943-437f2d500b1a" (UID: "b30c08ad-dcd8-400b-8943-437f2d500b1a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:25:41.661861 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:41.661798 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b30c08ad-dcd8-400b-8943-437f2d500b1a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b30c08ad-dcd8-400b-8943-437f2d500b1a" (UID: "b30c08ad-dcd8-400b-8943-437f2d500b1a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:25:41.663894 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:41.663872 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b30c08ad-dcd8-400b-8943-437f2d500b1a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b30c08ad-dcd8-400b-8943-437f2d500b1a" (UID: "b30c08ad-dcd8-400b-8943-437f2d500b1a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 18:25:41.664008 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:41.663890 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b30c08ad-dcd8-400b-8943-437f2d500b1a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b30c08ad-dcd8-400b-8943-437f2d500b1a" (UID: "b30c08ad-dcd8-400b-8943-437f2d500b1a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 18:25:41.664008 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:41.663896 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b30c08ad-dcd8-400b-8943-437f2d500b1a-kube-api-access-cz58h" (OuterVolumeSpecName: "kube-api-access-cz58h") pod "b30c08ad-dcd8-400b-8943-437f2d500b1a" (UID: "b30c08ad-dcd8-400b-8943-437f2d500b1a"). InnerVolumeSpecName "kube-api-access-cz58h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:25:41.762371 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:41.762337 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cz58h\" (UniqueName: \"kubernetes.io/projected/b30c08ad-dcd8-400b-8943-437f2d500b1a-kube-api-access-cz58h\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:25:41.762371 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:41.762368 2555 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b30c08ad-dcd8-400b-8943-437f2d500b1a-oauth-serving-cert\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:25:41.762371 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:41.762378 2555 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b30c08ad-dcd8-400b-8943-437f2d500b1a-service-ca\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:25:41.762616 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:41.762387 2555 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b30c08ad-dcd8-400b-8943-437f2d500b1a-console-oauth-config\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:25:41.762616 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:41.762395 2555 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b30c08ad-dcd8-400b-8943-437f2d500b1a-trusted-ca-bundle\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:25:41.762616 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:41.762403 2555 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b30c08ad-dcd8-400b-8943-437f2d500b1a-console-config\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:25:41.762616 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:41.762412 2555 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b30c08ad-dcd8-400b-8943-437f2d500b1a-console-serving-cert\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:25:41.939873 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:41.939782 2555 generic.go:356] "Generic (PLEG): container finished" podID="b30c08ad-dcd8-400b-8943-437f2d500b1a" containerID="e8299189b8ff484b89b459ade4cff936b4f3b79876e11dd2c20abffcc043bdfd" exitCode=0 Sep 08 18:25:41.939873 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:41.939824 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64d976c5fb-v7g8t" Sep 08 18:25:41.940055 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:41.939873 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64d976c5fb-v7g8t" event={"ID":"b30c08ad-dcd8-400b-8943-437f2d500b1a","Type":"ContainerDied","Data":"e8299189b8ff484b89b459ade4cff936b4f3b79876e11dd2c20abffcc043bdfd"} Sep 08 18:25:41.940055 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:41.939925 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64d976c5fb-v7g8t" event={"ID":"b30c08ad-dcd8-400b-8943-437f2d500b1a","Type":"ContainerDied","Data":"586d136cb8dfc0437d6d5fd5effeb5ee3e01ea80a3f7b6ad522dc8f025ca4b25"} Sep 08 18:25:41.940055 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:41.939946 2555 scope.go:117] "RemoveContainer" containerID="e8299189b8ff484b89b459ade4cff936b4f3b79876e11dd2c20abffcc043bdfd" Sep 08 18:25:41.948244 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:41.948226 2555 scope.go:117] "RemoveContainer" containerID="e8299189b8ff484b89b459ade4cff936b4f3b79876e11dd2c20abffcc043bdfd" Sep 08 18:25:41.948536 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:25:41.948519 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8299189b8ff484b89b459ade4cff936b4f3b79876e11dd2c20abffcc043bdfd\": container with ID starting with e8299189b8ff484b89b459ade4cff936b4f3b79876e11dd2c20abffcc043bdfd not found: ID does not exist" containerID="e8299189b8ff484b89b459ade4cff936b4f3b79876e11dd2c20abffcc043bdfd" Sep 08 18:25:41.948608 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:41.948543 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8299189b8ff484b89b459ade4cff936b4f3b79876e11dd2c20abffcc043bdfd"} err="failed to get container status \"e8299189b8ff484b89b459ade4cff936b4f3b79876e11dd2c20abffcc043bdfd\": rpc error: code = NotFound desc = could not find container \"e8299189b8ff484b89b459ade4cff936b4f3b79876e11dd2c20abffcc043bdfd\": container with ID starting with e8299189b8ff484b89b459ade4cff936b4f3b79876e11dd2c20abffcc043bdfd not found: ID does not exist" Sep 08 18:25:41.959544 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:41.959512 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64d976c5fb-v7g8t"] Sep 08 18:25:41.965918 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:41.965895 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-64d976c5fb-v7g8t"] Sep 08 18:25:42.323295 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:42.323260 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b30c08ad-dcd8-400b-8943-437f2d500b1a" path="/var/lib/kubelet/pods/b30c08ad-dcd8-400b-8943-437f2d500b1a/volumes" Sep 08 18:25:48.794252 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:48.794219 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:48.894794 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:48.894759 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:48.968728 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:48.968676 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:52.112547 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:52.112509 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:52.182748 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:52.182676 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:25:53.988101 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:53.988067 2555 generic.go:356] "Generic (PLEG): container finished" podID="41932272-e547-4229-be69-f99a4096637e" containerID="25c4e13bf7bd9e068a49832fe2515f595a75cf42202eaa69de710f9908323f5c" exitCode=0 Sep 08 18:25:53.988437 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:53.988114 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-flsvz" event={"ID":"41932272-e547-4229-be69-f99a4096637e","Type":"ContainerDied","Data":"25c4e13bf7bd9e068a49832fe2515f595a75cf42202eaa69de710f9908323f5c"} Sep 08 18:25:53.988480 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:53.988444 2555 scope.go:117] "RemoveContainer" containerID="25c4e13bf7bd9e068a49832fe2515f595a75cf42202eaa69de710f9908323f5c" Sep 08 18:25:54.993920 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:25:54.993885 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-flsvz" event={"ID":"41932272-e547-4229-be69-f99a4096637e","Type":"ContainerStarted","Data":"5dae3b40446e4281c0afdcc9b9cf09b02775f81eaef9db4e885eeaf1013faebf"} Sep 08 18:26:30.774156 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:30.774065 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:26:31.205565 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:31.205527 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:26:34.014529 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.014496 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 18:26:34.014950 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.014823 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b30c08ad-dcd8-400b-8943-437f2d500b1a" containerName="console" Sep 08 18:26:34.014950 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.014836 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="b30c08ad-dcd8-400b-8943-437f2d500b1a" containerName="console" Sep 08 18:26:34.014950 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.014847 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="612d5222-58c3-4da0-adc4-760f3fcd1843" containerName="registry" Sep 08 18:26:34.014950 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.014852 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="612d5222-58c3-4da0-adc4-760f3fcd1843" containerName="registry" Sep 08 18:26:34.014950 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.014921 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="b30c08ad-dcd8-400b-8943-437f2d500b1a" containerName="console" Sep 08 18:26:34.014950 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.014929 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="612d5222-58c3-4da0-adc4-760f3fcd1843" containerName="registry" Sep 08 18:26:34.018410 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.018392 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.024024 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.023996 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 08 18:26:34.024151 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.024128 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 08 18:26:34.024216 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.024145 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 08 18:26:34.024273 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.024248 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 08 18:26:34.024345 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.024330 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 08 18:26:34.024413 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.024398 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-xxcn7\"" Sep 08 18:26:34.024502 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.024474 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 08 18:26:34.024623 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.024519 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 08 18:26:34.024748 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.024731 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 08 18:26:34.028055 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.028037 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 08 18:26:34.034114 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.034094 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 18:26:34.063350 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.063326 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee96a90f-f261-4b6e-a081-0342c099e5a0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.063510 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.063357 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee96a90f-f261-4b6e-a081-0342c099e5a0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.063510 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.063377 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee96a90f-f261-4b6e-a081-0342c099e5a0-config-out\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.063510 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.063400 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee96a90f-f261-4b6e-a081-0342c099e5a0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.063510 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.063459 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee96a90f-f261-4b6e-a081-0342c099e5a0-web-config\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.063510 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.063484 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ee96a90f-f261-4b6e-a081-0342c099e5a0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.063773 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.063588 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ee96a90f-f261-4b6e-a081-0342c099e5a0-config-volume\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.063773 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.063619 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ee96a90f-f261-4b6e-a081-0342c099e5a0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.063773 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.063679 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mq6qn\" (UniqueName: \"kubernetes.io/projected/ee96a90f-f261-4b6e-a081-0342c099e5a0-kube-api-access-mq6qn\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.063773 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.063716 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ee96a90f-f261-4b6e-a081-0342c099e5a0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.063773 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.063731 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee96a90f-f261-4b6e-a081-0342c099e5a0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.063773 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.063752 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee96a90f-f261-4b6e-a081-0342c099e5a0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.063959 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.063777 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ee96a90f-f261-4b6e-a081-0342c099e5a0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.164102 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.164072 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mq6qn\" (UniqueName: \"kubernetes.io/projected/ee96a90f-f261-4b6e-a081-0342c099e5a0-kube-api-access-mq6qn\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.164102 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.164103 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ee96a90f-f261-4b6e-a081-0342c099e5a0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.164318 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.164120 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee96a90f-f261-4b6e-a081-0342c099e5a0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.164318 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.164137 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee96a90f-f261-4b6e-a081-0342c099e5a0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.164318 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.164166 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ee96a90f-f261-4b6e-a081-0342c099e5a0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.164318 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.164200 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee96a90f-f261-4b6e-a081-0342c099e5a0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.164318 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.164233 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee96a90f-f261-4b6e-a081-0342c099e5a0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.164318 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.164268 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee96a90f-f261-4b6e-a081-0342c099e5a0-config-out\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.164318 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.164316 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee96a90f-f261-4b6e-a081-0342c099e5a0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.164660 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.164348 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee96a90f-f261-4b6e-a081-0342c099e5a0-web-config\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.164660 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.164369 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ee96a90f-f261-4b6e-a081-0342c099e5a0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.164660 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.164407 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ee96a90f-f261-4b6e-a081-0342c099e5a0-config-volume\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.164660 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.164428 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ee96a90f-f261-4b6e-a081-0342c099e5a0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.165179 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.165153 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee96a90f-f261-4b6e-a081-0342c099e5a0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.165348 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.165165 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee96a90f-f261-4b6e-a081-0342c099e5a0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.165348 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.165234 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ee96a90f-f261-4b6e-a081-0342c099e5a0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.167578 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.167557 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee96a90f-f261-4b6e-a081-0342c099e5a0-config-out\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.167911 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.167888 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee96a90f-f261-4b6e-a081-0342c099e5a0-web-config\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.168128 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.168078 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ee96a90f-f261-4b6e-a081-0342c099e5a0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.168128 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.168104 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ee96a90f-f261-4b6e-a081-0342c099e5a0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.168297 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.168213 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee96a90f-f261-4b6e-a081-0342c099e5a0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.168346 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.168303 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ee96a90f-f261-4b6e-a081-0342c099e5a0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.168346 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.168323 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee96a90f-f261-4b6e-a081-0342c099e5a0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.168671 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.168654 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ee96a90f-f261-4b6e-a081-0342c099e5a0-config-volume\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.169122 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.169104 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee96a90f-f261-4b6e-a081-0342c099e5a0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.171607 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.171590 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mq6qn\" (UniqueName: \"kubernetes.io/projected/ee96a90f-f261-4b6e-a081-0342c099e5a0-kube-api-access-mq6qn\") pod \"alertmanager-main-0\" (UID: \"ee96a90f-f261-4b6e-a081-0342c099e5a0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.332014 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.331982 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 08 18:26:34.615576 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:34.615548 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 08 18:26:34.617051 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:26:34.617028 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee96a90f_f261_4b6e_a081_0342c099e5a0.slice/crio-431a513ead3751457bd80f9b34a814535b834cb3d3faac02e390d813e1fe6364 WatchSource:0}: Error finding container 431a513ead3751457bd80f9b34a814535b834cb3d3faac02e390d813e1fe6364: Status 404 returned error can't find the container with id 431a513ead3751457bd80f9b34a814535b834cb3d3faac02e390d813e1fe6364 Sep 08 18:26:35.151729 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:35.151671 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee96a90f-f261-4b6e-a081-0342c099e5a0","Type":"ContainerStarted","Data":"431a513ead3751457bd80f9b34a814535b834cb3d3faac02e390d813e1fe6364"} Sep 08 18:26:35.847441 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:35.847407 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6d85bb5d46-bxjxt"] Sep 08 18:26:35.852210 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:35.850946 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6d85bb5d46-bxjxt" Sep 08 18:26:35.853520 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:35.853490 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 08 18:26:35.853520 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:35.853507 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 08 18:26:35.853520 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:35.853516 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 08 18:26:35.853812 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:35.853504 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 08 18:26:35.853911 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:35.853897 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-rtgm8\"" Sep 08 18:26:35.854009 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:35.853988 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 08 18:26:35.858886 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:35.858857 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 08 18:26:35.866270 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:35.866247 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6d85bb5d46-bxjxt"] Sep 08 18:26:35.877466 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:35.877430 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96fcab03-2331-4636-881c-eb6292fe6243-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6d85bb5d46-bxjxt\" (UID: \"96fcab03-2331-4636-881c-eb6292fe6243\") " pod="openshift-monitoring/telemeter-client-6d85bb5d46-bxjxt" Sep 08 18:26:35.877589 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:35.877480 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/96fcab03-2331-4636-881c-eb6292fe6243-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6d85bb5d46-bxjxt\" (UID: \"96fcab03-2331-4636-881c-eb6292fe6243\") " pod="openshift-monitoring/telemeter-client-6d85bb5d46-bxjxt" Sep 08 18:26:35.877589 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:35.877563 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-md5nm\" (UniqueName: \"kubernetes.io/projected/96fcab03-2331-4636-881c-eb6292fe6243-kube-api-access-md5nm\") pod \"telemeter-client-6d85bb5d46-bxjxt\" (UID: \"96fcab03-2331-4636-881c-eb6292fe6243\") " pod="openshift-monitoring/telemeter-client-6d85bb5d46-bxjxt" Sep 08 18:26:35.877677 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:35.877598 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/96fcab03-2331-4636-881c-eb6292fe6243-federate-client-tls\") pod \"telemeter-client-6d85bb5d46-bxjxt\" (UID: \"96fcab03-2331-4636-881c-eb6292fe6243\") " pod="openshift-monitoring/telemeter-client-6d85bb5d46-bxjxt" Sep 08 18:26:35.877747 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:35.877733 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96fcab03-2331-4636-881c-eb6292fe6243-metrics-client-ca\") pod \"telemeter-client-6d85bb5d46-bxjxt\" (UID: \"96fcab03-2331-4636-881c-eb6292fe6243\") " pod="openshift-monitoring/telemeter-client-6d85bb5d46-bxjxt" Sep 08 18:26:35.877787 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:35.877757 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/96fcab03-2331-4636-881c-eb6292fe6243-secret-telemeter-client\") pod \"telemeter-client-6d85bb5d46-bxjxt\" (UID: \"96fcab03-2331-4636-881c-eb6292fe6243\") " pod="openshift-monitoring/telemeter-client-6d85bb5d46-bxjxt" Sep 08 18:26:35.877860 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:35.877837 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96fcab03-2331-4636-881c-eb6292fe6243-serving-certs-ca-bundle\") pod \"telemeter-client-6d85bb5d46-bxjxt\" (UID: \"96fcab03-2331-4636-881c-eb6292fe6243\") " pod="openshift-monitoring/telemeter-client-6d85bb5d46-bxjxt" Sep 08 18:26:35.877925 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:35.877910 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/96fcab03-2331-4636-881c-eb6292fe6243-telemeter-client-tls\") pod \"telemeter-client-6d85bb5d46-bxjxt\" (UID: \"96fcab03-2331-4636-881c-eb6292fe6243\") " pod="openshift-monitoring/telemeter-client-6d85bb5d46-bxjxt" Sep 08 18:26:35.979189 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:35.979090 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-md5nm\" (UniqueName: \"kubernetes.io/projected/96fcab03-2331-4636-881c-eb6292fe6243-kube-api-access-md5nm\") pod \"telemeter-client-6d85bb5d46-bxjxt\" (UID: \"96fcab03-2331-4636-881c-eb6292fe6243\") " pod="openshift-monitoring/telemeter-client-6d85bb5d46-bxjxt" Sep 08 18:26:35.979189 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:35.979149 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/96fcab03-2331-4636-881c-eb6292fe6243-federate-client-tls\") pod \"telemeter-client-6d85bb5d46-bxjxt\" (UID: \"96fcab03-2331-4636-881c-eb6292fe6243\") " pod="openshift-monitoring/telemeter-client-6d85bb5d46-bxjxt" Sep 08 18:26:35.979418 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:35.979207 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96fcab03-2331-4636-881c-eb6292fe6243-metrics-client-ca\") pod \"telemeter-client-6d85bb5d46-bxjxt\" (UID: \"96fcab03-2331-4636-881c-eb6292fe6243\") " pod="openshift-monitoring/telemeter-client-6d85bb5d46-bxjxt" Sep 08 18:26:35.979418 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:35.979235 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/96fcab03-2331-4636-881c-eb6292fe6243-secret-telemeter-client\") pod \"telemeter-client-6d85bb5d46-bxjxt\" (UID: \"96fcab03-2331-4636-881c-eb6292fe6243\") " pod="openshift-monitoring/telemeter-client-6d85bb5d46-bxjxt" Sep 08 18:26:35.979418 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:35.979271 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96fcab03-2331-4636-881c-eb6292fe6243-serving-certs-ca-bundle\") pod \"telemeter-client-6d85bb5d46-bxjxt\" (UID: \"96fcab03-2331-4636-881c-eb6292fe6243\") " pod="openshift-monitoring/telemeter-client-6d85bb5d46-bxjxt" Sep 08 18:26:35.979418 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:35.979322 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/96fcab03-2331-4636-881c-eb6292fe6243-telemeter-client-tls\") pod \"telemeter-client-6d85bb5d46-bxjxt\" (UID: \"96fcab03-2331-4636-881c-eb6292fe6243\") " pod="openshift-monitoring/telemeter-client-6d85bb5d46-bxjxt" Sep 08 18:26:35.979418 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:35.979357 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96fcab03-2331-4636-881c-eb6292fe6243-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6d85bb5d46-bxjxt\" (UID: \"96fcab03-2331-4636-881c-eb6292fe6243\") " pod="openshift-monitoring/telemeter-client-6d85bb5d46-bxjxt" Sep 08 18:26:35.979418 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:35.979389 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/96fcab03-2331-4636-881c-eb6292fe6243-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6d85bb5d46-bxjxt\" (UID: \"96fcab03-2331-4636-881c-eb6292fe6243\") " pod="openshift-monitoring/telemeter-client-6d85bb5d46-bxjxt" Sep 08 18:26:35.980109 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:35.980056 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96fcab03-2331-4636-881c-eb6292fe6243-metrics-client-ca\") pod \"telemeter-client-6d85bb5d46-bxjxt\" (UID: \"96fcab03-2331-4636-881c-eb6292fe6243\") " pod="openshift-monitoring/telemeter-client-6d85bb5d46-bxjxt" Sep 08 18:26:35.980109 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:35.980071 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96fcab03-2331-4636-881c-eb6292fe6243-serving-certs-ca-bundle\") pod \"telemeter-client-6d85bb5d46-bxjxt\" (UID: \"96fcab03-2331-4636-881c-eb6292fe6243\") " pod="openshift-monitoring/telemeter-client-6d85bb5d46-bxjxt" Sep 08 18:26:35.980374 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:35.980350 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96fcab03-2331-4636-881c-eb6292fe6243-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6d85bb5d46-bxjxt\" (UID: \"96fcab03-2331-4636-881c-eb6292fe6243\") " pod="openshift-monitoring/telemeter-client-6d85bb5d46-bxjxt" Sep 08 18:26:35.982632 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:35.982607 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/96fcab03-2331-4636-881c-eb6292fe6243-secret-telemeter-client\") pod \"telemeter-client-6d85bb5d46-bxjxt\" (UID: \"96fcab03-2331-4636-881c-eb6292fe6243\") " pod="openshift-monitoring/telemeter-client-6d85bb5d46-bxjxt" Sep 08 18:26:35.982778 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:35.982649 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/96fcab03-2331-4636-881c-eb6292fe6243-federate-client-tls\") pod \"telemeter-client-6d85bb5d46-bxjxt\" (UID: \"96fcab03-2331-4636-881c-eb6292fe6243\") " pod="openshift-monitoring/telemeter-client-6d85bb5d46-bxjxt" Sep 08 18:26:35.982778 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:35.982670 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/96fcab03-2331-4636-881c-eb6292fe6243-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6d85bb5d46-bxjxt\" (UID: \"96fcab03-2331-4636-881c-eb6292fe6243\") " pod="openshift-monitoring/telemeter-client-6d85bb5d46-bxjxt" Sep 08 18:26:35.983028 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:35.983009 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/96fcab03-2331-4636-881c-eb6292fe6243-telemeter-client-tls\") pod \"telemeter-client-6d85bb5d46-bxjxt\" (UID: \"96fcab03-2331-4636-881c-eb6292fe6243\") " pod="openshift-monitoring/telemeter-client-6d85bb5d46-bxjxt" Sep 08 18:26:35.989058 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:35.989033 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-md5nm\" (UniqueName: \"kubernetes.io/projected/96fcab03-2331-4636-881c-eb6292fe6243-kube-api-access-md5nm\") pod \"telemeter-client-6d85bb5d46-bxjxt\" (UID: \"96fcab03-2331-4636-881c-eb6292fe6243\") " pod="openshift-monitoring/telemeter-client-6d85bb5d46-bxjxt" Sep 08 18:26:36.156472 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:36.156438 2555 generic.go:356] "Generic (PLEG): container finished" podID="ee96a90f-f261-4b6e-a081-0342c099e5a0" containerID="edb4289338d7b16d163e43708e5ecfab16f70598aef6166b6d72a5fe420a9990" exitCode=0 Sep 08 18:26:36.156861 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:36.156530 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee96a90f-f261-4b6e-a081-0342c099e5a0","Type":"ContainerDied","Data":"edb4289338d7b16d163e43708e5ecfab16f70598aef6166b6d72a5fe420a9990"} Sep 08 18:26:36.162253 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:36.162232 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6d85bb5d46-bxjxt" Sep 08 18:26:36.457364 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:36.457324 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6d85bb5d46-bxjxt"] Sep 08 18:26:36.460594 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:26:36.460565 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96fcab03_2331_4636_881c_eb6292fe6243.slice/crio-caad1c293f649b71c4aea2bb77f1e19144836c9014c1db6c4a412408b135aba5 WatchSource:0}: Error finding container caad1c293f649b71c4aea2bb77f1e19144836c9014c1db6c4a412408b135aba5: Status 404 returned error can't find the container with id caad1c293f649b71c4aea2bb77f1e19144836c9014c1db6c4a412408b135aba5 Sep 08 18:26:37.163130 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:37.163093 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d85bb5d46-bxjxt" event={"ID":"96fcab03-2331-4636-881c-eb6292fe6243","Type":"ContainerStarted","Data":"caad1c293f649b71c4aea2bb77f1e19144836c9014c1db6c4a412408b135aba5"} Sep 08 18:26:39.178827 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.178778 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d85bb5d46-bxjxt" event={"ID":"96fcab03-2331-4636-881c-eb6292fe6243","Type":"ContainerStarted","Data":"605cb287ca1a5c7a891d14d4467ba81a428b75d99792c42a53ac66e23fb13e35"} Sep 08 18:26:39.179288 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.178880 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d85bb5d46-bxjxt" event={"ID":"96fcab03-2331-4636-881c-eb6292fe6243","Type":"ContainerStarted","Data":"c24eed38e959195f7132bb6543b3de22be9c966a8c256a6e055cc917089d6f26"} Sep 08 18:26:39.179288 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.178906 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d85bb5d46-bxjxt" event={"ID":"96fcab03-2331-4636-881c-eb6292fe6243","Type":"ContainerStarted","Data":"7cb1bdcfa34e30a594ac5b0c737d963485d969afa6f4586ddb03ee4b904a1d0b"} Sep 08 18:26:39.182226 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.182203 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee96a90f-f261-4b6e-a081-0342c099e5a0","Type":"ContainerStarted","Data":"e1e5841c6bb36e7768c1266c7e5fa37e5592042c0abd9fbed290a9618a48bda3"} Sep 08 18:26:39.182356 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.182253 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee96a90f-f261-4b6e-a081-0342c099e5a0","Type":"ContainerStarted","Data":"66865b3fb4936fead46b3068c60216506e15ec53951a37502a19156d9324d8d9"} Sep 08 18:26:39.182356 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.182278 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee96a90f-f261-4b6e-a081-0342c099e5a0","Type":"ContainerStarted","Data":"2b9a822a626ac3282426eea69bb486d68bdcdcf2994e6f7074d4fb7f48843a8c"} Sep 08 18:26:39.182356 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.182301 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee96a90f-f261-4b6e-a081-0342c099e5a0","Type":"ContainerStarted","Data":"f0edddf5d4cc60fdba527c0918be0342ca9f684a8127dddad7fb0a633e39dbe5"} Sep 08 18:26:39.182356 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.182329 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee96a90f-f261-4b6e-a081-0342c099e5a0","Type":"ContainerStarted","Data":"aff2696977ece2419b0e3ca94f1a8894e6925f7593e72e6124deb9578ff6965a"} Sep 08 18:26:39.182356 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.182353 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee96a90f-f261-4b6e-a081-0342c099e5a0","Type":"ContainerStarted","Data":"5949660548e9f638e0b04d882016d063c859efd284c2e339e01da993d5a155bc"} Sep 08 18:26:39.201794 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.201744 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6d85bb5d46-bxjxt" podStartSLOduration=2.218669489 podStartE2EDuration="4.201731047s" totalImagesPullingTime="1.983061558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:26:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:26:39.19965784 +0000 UTC m=+181.405643236" watchObservedRunningTime="2026-09-08 18:26:39.201731047 +0000 UTC m=+181.407716444" Sep 08 18:26:39.225488 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.225435 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.901337795 podStartE2EDuration="6.225417353s" totalImagesPullingTime="3.307879578s" totalInitContainerRuntime="16.19998ms" isStatefulPod=false podCreationTimestamp="2026-09-08 18:26:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:26:39.224087876 +0000 UTC m=+181.430073284" watchObservedRunningTime="2026-09-08 18:26:39.225417353 +0000 UTC m=+181.431402754" Sep 08 18:26:39.291614 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.291575 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-76ff479d85-78l4q"] Sep 08 18:26:39.295314 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.295284 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76ff479d85-78l4q" Sep 08 18:26:39.297595 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.297314 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 08 18:26:39.297595 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.297417 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 08 18:26:39.297595 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.297454 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-vdqzb\"" Sep 08 18:26:39.297595 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.297490 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 08 18:26:39.297595 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.297556 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 08 18:26:39.298143 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.297715 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 08 18:26:39.302860 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.302821 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 08 18:26:39.304526 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.304507 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76ff479d85-78l4q"] Sep 08 18:26:39.349065 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.349034 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:26:39.403706 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.403642 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:26:39.410842 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.410823 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c76daa71-8efb-452c-806f-00c68893eb2b-console-config\") pod \"console-76ff479d85-78l4q\" (UID: \"c76daa71-8efb-452c-806f-00c68893eb2b\") " pod="openshift-console/console-76ff479d85-78l4q" Sep 08 18:26:39.410912 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.410859 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c76daa71-8efb-452c-806f-00c68893eb2b-trusted-ca-bundle\") pod \"console-76ff479d85-78l4q\" (UID: \"c76daa71-8efb-452c-806f-00c68893eb2b\") " pod="openshift-console/console-76ff479d85-78l4q" Sep 08 18:26:39.410959 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.410913 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c76daa71-8efb-452c-806f-00c68893eb2b-service-ca\") pod \"console-76ff479d85-78l4q\" (UID: \"c76daa71-8efb-452c-806f-00c68893eb2b\") " pod="openshift-console/console-76ff479d85-78l4q" Sep 08 18:26:39.410959 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.410938 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c76daa71-8efb-452c-806f-00c68893eb2b-console-serving-cert\") pod \"console-76ff479d85-78l4q\" (UID: \"c76daa71-8efb-452c-806f-00c68893eb2b\") " pod="openshift-console/console-76ff479d85-78l4q" Sep 08 18:26:39.411035 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.410959 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c76daa71-8efb-452c-806f-00c68893eb2b-oauth-serving-cert\") pod \"console-76ff479d85-78l4q\" (UID: \"c76daa71-8efb-452c-806f-00c68893eb2b\") " pod="openshift-console/console-76ff479d85-78l4q" Sep 08 18:26:39.411035 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.411007 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c76daa71-8efb-452c-806f-00c68893eb2b-console-oauth-config\") pod \"console-76ff479d85-78l4q\" (UID: \"c76daa71-8efb-452c-806f-00c68893eb2b\") " pod="openshift-console/console-76ff479d85-78l4q" Sep 08 18:26:39.411108 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.411037 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kk4b\" (UniqueName: \"kubernetes.io/projected/c76daa71-8efb-452c-806f-00c68893eb2b-kube-api-access-6kk4b\") pod \"console-76ff479d85-78l4q\" (UID: \"c76daa71-8efb-452c-806f-00c68893eb2b\") " pod="openshift-console/console-76ff479d85-78l4q" Sep 08 18:26:39.511842 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.511768 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c76daa71-8efb-452c-806f-00c68893eb2b-service-ca\") pod \"console-76ff479d85-78l4q\" (UID: \"c76daa71-8efb-452c-806f-00c68893eb2b\") " pod="openshift-console/console-76ff479d85-78l4q" Sep 08 18:26:39.511842 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.511803 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c76daa71-8efb-452c-806f-00c68893eb2b-console-serving-cert\") pod \"console-76ff479d85-78l4q\" (UID: \"c76daa71-8efb-452c-806f-00c68893eb2b\") " pod="openshift-console/console-76ff479d85-78l4q" Sep 08 18:26:39.511842 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.511823 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c76daa71-8efb-452c-806f-00c68893eb2b-oauth-serving-cert\") pod \"console-76ff479d85-78l4q\" (UID: \"c76daa71-8efb-452c-806f-00c68893eb2b\") " pod="openshift-console/console-76ff479d85-78l4q" Sep 08 18:26:39.512067 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.511856 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c76daa71-8efb-452c-806f-00c68893eb2b-console-oauth-config\") pod \"console-76ff479d85-78l4q\" (UID: \"c76daa71-8efb-452c-806f-00c68893eb2b\") " pod="openshift-console/console-76ff479d85-78l4q" Sep 08 18:26:39.512067 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.511886 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kk4b\" (UniqueName: \"kubernetes.io/projected/c76daa71-8efb-452c-806f-00c68893eb2b-kube-api-access-6kk4b\") pod \"console-76ff479d85-78l4q\" (UID: \"c76daa71-8efb-452c-806f-00c68893eb2b\") " pod="openshift-console/console-76ff479d85-78l4q" Sep 08 18:26:39.512067 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.511916 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c76daa71-8efb-452c-806f-00c68893eb2b-console-config\") pod \"console-76ff479d85-78l4q\" (UID: \"c76daa71-8efb-452c-806f-00c68893eb2b\") " pod="openshift-console/console-76ff479d85-78l4q" Sep 08 18:26:39.512067 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.511938 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c76daa71-8efb-452c-806f-00c68893eb2b-trusted-ca-bundle\") pod \"console-76ff479d85-78l4q\" (UID: \"c76daa71-8efb-452c-806f-00c68893eb2b\") " pod="openshift-console/console-76ff479d85-78l4q" Sep 08 18:26:39.512571 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.512549 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c76daa71-8efb-452c-806f-00c68893eb2b-service-ca\") pod \"console-76ff479d85-78l4q\" (UID: \"c76daa71-8efb-452c-806f-00c68893eb2b\") " pod="openshift-console/console-76ff479d85-78l4q" Sep 08 18:26:39.512645 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.512582 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c76daa71-8efb-452c-806f-00c68893eb2b-console-config\") pod \"console-76ff479d85-78l4q\" (UID: \"c76daa71-8efb-452c-806f-00c68893eb2b\") " pod="openshift-console/console-76ff479d85-78l4q" Sep 08 18:26:39.512645 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.512552 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c76daa71-8efb-452c-806f-00c68893eb2b-oauth-serving-cert\") pod \"console-76ff479d85-78l4q\" (UID: \"c76daa71-8efb-452c-806f-00c68893eb2b\") " pod="openshift-console/console-76ff479d85-78l4q" Sep 08 18:26:39.512972 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.512952 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c76daa71-8efb-452c-806f-00c68893eb2b-trusted-ca-bundle\") pod \"console-76ff479d85-78l4q\" (UID: \"c76daa71-8efb-452c-806f-00c68893eb2b\") " pod="openshift-console/console-76ff479d85-78l4q" Sep 08 18:26:39.514585 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.514571 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c76daa71-8efb-452c-806f-00c68893eb2b-console-serving-cert\") pod \"console-76ff479d85-78l4q\" (UID: \"c76daa71-8efb-452c-806f-00c68893eb2b\") " pod="openshift-console/console-76ff479d85-78l4q" Sep 08 18:26:39.514633 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.514609 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c76daa71-8efb-452c-806f-00c68893eb2b-console-oauth-config\") pod \"console-76ff479d85-78l4q\" (UID: \"c76daa71-8efb-452c-806f-00c68893eb2b\") " pod="openshift-console/console-76ff479d85-78l4q" Sep 08 18:26:39.519718 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.519678 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kk4b\" (UniqueName: \"kubernetes.io/projected/c76daa71-8efb-452c-806f-00c68893eb2b-kube-api-access-6kk4b\") pod \"console-76ff479d85-78l4q\" (UID: \"c76daa71-8efb-452c-806f-00c68893eb2b\") " pod="openshift-console/console-76ff479d85-78l4q" Sep 08 18:26:39.607228 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.607190 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76ff479d85-78l4q" Sep 08 18:26:39.881538 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:39.881515 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76ff479d85-78l4q"] Sep 08 18:26:39.883527 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:26:39.883493 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc76daa71_8efb_452c_806f_00c68893eb2b.slice/crio-936c4229d62b249c47fd94757de6007809a0bb42bdae4948ad12e66076f8c2f0 WatchSource:0}: Error finding container 936c4229d62b249c47fd94757de6007809a0bb42bdae4948ad12e66076f8c2f0: Status 404 returned error can't find the container with id 936c4229d62b249c47fd94757de6007809a0bb42bdae4948ad12e66076f8c2f0 Sep 08 18:26:40.190280 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:40.190163 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76ff479d85-78l4q" event={"ID":"c76daa71-8efb-452c-806f-00c68893eb2b","Type":"ContainerStarted","Data":"db4dfd41a383f723594a1034ccfb2e1fa6d3b8b343f8076394bcaf5530084850"} Sep 08 18:26:40.190839 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:40.190800 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76ff479d85-78l4q" event={"ID":"c76daa71-8efb-452c-806f-00c68893eb2b","Type":"ContainerStarted","Data":"936c4229d62b249c47fd94757de6007809a0bb42bdae4948ad12e66076f8c2f0"} Sep 08 18:26:40.207889 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:40.207827 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-76ff479d85-78l4q" podStartSLOduration=1.207810908 podStartE2EDuration="1.207810908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:26:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:26:40.20696708 +0000 UTC m=+182.412952483" watchObservedRunningTime="2026-09-08 18:26:40.207810908 +0000 UTC m=+182.413796304" Sep 08 18:26:41.461548 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:41.461508 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:26:43.671180 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:43.671143 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:26:49.607781 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:49.607737 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-76ff479d85-78l4q" Sep 08 18:26:49.608258 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:49.607843 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76ff479d85-78l4q" Sep 08 18:26:49.612427 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:49.612406 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-76ff479d85-78l4q" Sep 08 18:26:50.232219 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:50.232189 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-76ff479d85-78l4q" Sep 08 18:26:50.395242 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:50.395208 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:26:50.462469 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:50.462440 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:26:50.517953 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:26:50.517869 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:27:24.994977 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:27:24.994942 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:27:25.095165 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:27:25.095106 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:27:46.016987 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:27:46.016953 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:27:46.075861 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:27:46.075838 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:27:53.563972 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:27:53.563934 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-85cd9df9bd-pgq8c"] Sep 08 18:27:53.568436 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:27:53.568409 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85cd9df9bd-pgq8c" Sep 08 18:27:53.577318 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:27:53.577290 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85cd9df9bd-pgq8c"] Sep 08 18:27:53.601645 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:27:53.601611 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whqkx\" (UniqueName: \"kubernetes.io/projected/4fd783b8-b4a1-4f7d-a308-49e85f4be430-kube-api-access-whqkx\") pod \"console-85cd9df9bd-pgq8c\" (UID: \"4fd783b8-b4a1-4f7d-a308-49e85f4be430\") " pod="openshift-console/console-85cd9df9bd-pgq8c" Sep 08 18:27:53.601645 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:27:53.601654 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fd783b8-b4a1-4f7d-a308-49e85f4be430-trusted-ca-bundle\") pod \"console-85cd9df9bd-pgq8c\" (UID: \"4fd783b8-b4a1-4f7d-a308-49e85f4be430\") " pod="openshift-console/console-85cd9df9bd-pgq8c" Sep 08 18:27:53.601866 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:27:53.601677 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4fd783b8-b4a1-4f7d-a308-49e85f4be430-console-oauth-config\") pod \"console-85cd9df9bd-pgq8c\" (UID: \"4fd783b8-b4a1-4f7d-a308-49e85f4be430\") " pod="openshift-console/console-85cd9df9bd-pgq8c" Sep 08 18:27:53.601866 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:27:53.601736 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4fd783b8-b4a1-4f7d-a308-49e85f4be430-console-serving-cert\") pod \"console-85cd9df9bd-pgq8c\" (UID: \"4fd783b8-b4a1-4f7d-a308-49e85f4be430\") " pod="openshift-console/console-85cd9df9bd-pgq8c" Sep 08 18:27:53.601866 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:27:53.601780 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4fd783b8-b4a1-4f7d-a308-49e85f4be430-oauth-serving-cert\") pod \"console-85cd9df9bd-pgq8c\" (UID: \"4fd783b8-b4a1-4f7d-a308-49e85f4be430\") " pod="openshift-console/console-85cd9df9bd-pgq8c" Sep 08 18:27:53.601866 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:27:53.601801 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4fd783b8-b4a1-4f7d-a308-49e85f4be430-console-config\") pod \"console-85cd9df9bd-pgq8c\" (UID: \"4fd783b8-b4a1-4f7d-a308-49e85f4be430\") " pod="openshift-console/console-85cd9df9bd-pgq8c" Sep 08 18:27:53.601866 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:27:53.601815 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4fd783b8-b4a1-4f7d-a308-49e85f4be430-service-ca\") pod \"console-85cd9df9bd-pgq8c\" (UID: \"4fd783b8-b4a1-4f7d-a308-49e85f4be430\") " pod="openshift-console/console-85cd9df9bd-pgq8c" Sep 08 18:27:53.653634 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:27:53.653603 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:27:53.702491 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:27:53.702448 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whqkx\" (UniqueName: \"kubernetes.io/projected/4fd783b8-b4a1-4f7d-a308-49e85f4be430-kube-api-access-whqkx\") pod \"console-85cd9df9bd-pgq8c\" (UID: \"4fd783b8-b4a1-4f7d-a308-49e85f4be430\") " pod="openshift-console/console-85cd9df9bd-pgq8c" Sep 08 18:27:53.702491 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:27:53.702495 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fd783b8-b4a1-4f7d-a308-49e85f4be430-trusted-ca-bundle\") pod \"console-85cd9df9bd-pgq8c\" (UID: \"4fd783b8-b4a1-4f7d-a308-49e85f4be430\") " pod="openshift-console/console-85cd9df9bd-pgq8c" Sep 08 18:27:53.702745 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:27:53.702541 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4fd783b8-b4a1-4f7d-a308-49e85f4be430-console-oauth-config\") pod \"console-85cd9df9bd-pgq8c\" (UID: \"4fd783b8-b4a1-4f7d-a308-49e85f4be430\") " pod="openshift-console/console-85cd9df9bd-pgq8c" Sep 08 18:27:53.702745 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:27:53.702565 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4fd783b8-b4a1-4f7d-a308-49e85f4be430-console-serving-cert\") pod \"console-85cd9df9bd-pgq8c\" (UID: \"4fd783b8-b4a1-4f7d-a308-49e85f4be430\") " pod="openshift-console/console-85cd9df9bd-pgq8c" Sep 08 18:27:53.702745 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:27:53.702622 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4fd783b8-b4a1-4f7d-a308-49e85f4be430-oauth-serving-cert\") pod \"console-85cd9df9bd-pgq8c\" (UID: \"4fd783b8-b4a1-4f7d-a308-49e85f4be430\") " pod="openshift-console/console-85cd9df9bd-pgq8c" Sep 08 18:27:53.702745 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:27:53.702650 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4fd783b8-b4a1-4f7d-a308-49e85f4be430-console-config\") pod \"console-85cd9df9bd-pgq8c\" (UID: \"4fd783b8-b4a1-4f7d-a308-49e85f4be430\") " pod="openshift-console/console-85cd9df9bd-pgq8c" Sep 08 18:27:53.702745 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:27:53.702672 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4fd783b8-b4a1-4f7d-a308-49e85f4be430-service-ca\") pod \"console-85cd9df9bd-pgq8c\" (UID: \"4fd783b8-b4a1-4f7d-a308-49e85f4be430\") " pod="openshift-console/console-85cd9df9bd-pgq8c" Sep 08 18:27:53.703458 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:27:53.703432 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4fd783b8-b4a1-4f7d-a308-49e85f4be430-oauth-serving-cert\") pod \"console-85cd9df9bd-pgq8c\" (UID: \"4fd783b8-b4a1-4f7d-a308-49e85f4be430\") " pod="openshift-console/console-85cd9df9bd-pgq8c" Sep 08 18:27:53.703582 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:27:53.703521 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4fd783b8-b4a1-4f7d-a308-49e85f4be430-console-config\") pod \"console-85cd9df9bd-pgq8c\" (UID: \"4fd783b8-b4a1-4f7d-a308-49e85f4be430\") " pod="openshift-console/console-85cd9df9bd-pgq8c" Sep 08 18:27:53.703741 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:27:53.703717 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4fd783b8-b4a1-4f7d-a308-49e85f4be430-service-ca\") pod \"console-85cd9df9bd-pgq8c\" (UID: \"4fd783b8-b4a1-4f7d-a308-49e85f4be430\") " pod="openshift-console/console-85cd9df9bd-pgq8c" Sep 08 18:27:53.704044 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:27:53.704021 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fd783b8-b4a1-4f7d-a308-49e85f4be430-trusted-ca-bundle\") pod \"console-85cd9df9bd-pgq8c\" (UID: \"4fd783b8-b4a1-4f7d-a308-49e85f4be430\") " pod="openshift-console/console-85cd9df9bd-pgq8c" Sep 08 18:27:53.705241 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:27:53.705221 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4fd783b8-b4a1-4f7d-a308-49e85f4be430-console-serving-cert\") pod \"console-85cd9df9bd-pgq8c\" (UID: \"4fd783b8-b4a1-4f7d-a308-49e85f4be430\") " pod="openshift-console/console-85cd9df9bd-pgq8c" Sep 08 18:27:53.705331 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:27:53.705278 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4fd783b8-b4a1-4f7d-a308-49e85f4be430-console-oauth-config\") pod \"console-85cd9df9bd-pgq8c\" (UID: \"4fd783b8-b4a1-4f7d-a308-49e85f4be430\") " pod="openshift-console/console-85cd9df9bd-pgq8c" Sep 08 18:27:53.710644 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:27:53.710624 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whqkx\" (UniqueName: \"kubernetes.io/projected/4fd783b8-b4a1-4f7d-a308-49e85f4be430-kube-api-access-whqkx\") pod \"console-85cd9df9bd-pgq8c\" (UID: \"4fd783b8-b4a1-4f7d-a308-49e85f4be430\") " pod="openshift-console/console-85cd9df9bd-pgq8c" Sep 08 18:27:53.879986 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:27:53.879903 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85cd9df9bd-pgq8c" Sep 08 18:27:54.155045 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:27:54.154963 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85cd9df9bd-pgq8c"] Sep 08 18:27:54.159037 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:27:54.159011 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fd783b8_b4a1_4f7d_a308_49e85f4be430.slice/crio-682e44f239c32ea4c84400d566df935abc0565d54250092fa78fa41d509d6c09 WatchSource:0}: Error finding container 682e44f239c32ea4c84400d566df935abc0565d54250092fa78fa41d509d6c09: Status 404 returned error can't find the container with id 682e44f239c32ea4c84400d566df935abc0565d54250092fa78fa41d509d6c09 Sep 08 18:27:54.476505 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:27:54.476405 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85cd9df9bd-pgq8c" event={"ID":"4fd783b8-b4a1-4f7d-a308-49e85f4be430","Type":"ContainerStarted","Data":"73fbd2561a4da214089f3ddccb552abb7192021a73e3e2f8e68e00b638b30b1e"} Sep 08 18:27:54.476505 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:27:54.476456 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85cd9df9bd-pgq8c" event={"ID":"4fd783b8-b4a1-4f7d-a308-49e85f4be430","Type":"ContainerStarted","Data":"682e44f239c32ea4c84400d566df935abc0565d54250092fa78fa41d509d6c09"} Sep 08 18:27:54.495248 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:27:54.495191 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-85cd9df9bd-pgq8c" podStartSLOduration=1.495175372 podStartE2EDuration="1.495175372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:27:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:27:54.493562302 +0000 UTC m=+256.699547698" watchObservedRunningTime="2026-09-08 18:27:54.495175372 +0000 UTC m=+256.701160768" Sep 08 18:27:57.960271 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:27:57.960233 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:27:58.077751 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:27:58.077718 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:28:01.835429 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:01.835340 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:28:03.880447 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:03.880410 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-85cd9df9bd-pgq8c" Sep 08 18:28:03.880893 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:03.880477 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-85cd9df9bd-pgq8c" Sep 08 18:28:03.885120 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:03.885097 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-85cd9df9bd-pgq8c" Sep 08 18:28:03.936453 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:03.936415 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:28:04.037566 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:04.037507 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:28:04.521655 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:04.521631 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-85cd9df9bd-pgq8c" Sep 08 18:28:04.563961 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:04.563928 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76ff479d85-78l4q"] Sep 08 18:28:04.677560 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:04.677526 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:28:04.777925 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:04.777845 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:28:04.837112 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:04.837082 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:28:29.590849 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:29.590800 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-76ff479d85-78l4q" podUID="c76daa71-8efb-452c-806f-00c68893eb2b" containerName="console" containerID="cri-o://db4dfd41a383f723594a1034ccfb2e1fa6d3b8b343f8076394bcaf5530084850" gracePeriod=15 Sep 08 18:28:29.833846 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:29.833817 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76ff479d85-78l4q" Sep 08 18:28:29.912488 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:29.912390 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c76daa71-8efb-452c-806f-00c68893eb2b-oauth-serving-cert\") pod \"c76daa71-8efb-452c-806f-00c68893eb2b\" (UID: \"c76daa71-8efb-452c-806f-00c68893eb2b\") " Sep 08 18:28:29.912488 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:29.912426 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c76daa71-8efb-452c-806f-00c68893eb2b-service-ca\") pod \"c76daa71-8efb-452c-806f-00c68893eb2b\" (UID: \"c76daa71-8efb-452c-806f-00c68893eb2b\") " Sep 08 18:28:29.912488 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:29.912458 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c76daa71-8efb-452c-806f-00c68893eb2b-console-oauth-config\") pod \"c76daa71-8efb-452c-806f-00c68893eb2b\" (UID: \"c76daa71-8efb-452c-806f-00c68893eb2b\") " Sep 08 18:28:29.912488 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:29.912479 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c76daa71-8efb-452c-806f-00c68893eb2b-console-config\") pod \"c76daa71-8efb-452c-806f-00c68893eb2b\" (UID: \"c76daa71-8efb-452c-806f-00c68893eb2b\") " Sep 08 18:28:29.912863 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:29.912639 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6kk4b\" (UniqueName: \"kubernetes.io/projected/c76daa71-8efb-452c-806f-00c68893eb2b-kube-api-access-6kk4b\") pod \"c76daa71-8efb-452c-806f-00c68893eb2b\" (UID: \"c76daa71-8efb-452c-806f-00c68893eb2b\") " Sep 08 18:28:29.912863 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:29.912725 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c76daa71-8efb-452c-806f-00c68893eb2b-trusted-ca-bundle\") pod \"c76daa71-8efb-452c-806f-00c68893eb2b\" (UID: \"c76daa71-8efb-452c-806f-00c68893eb2b\") " Sep 08 18:28:29.912863 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:29.912760 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c76daa71-8efb-452c-806f-00c68893eb2b-console-serving-cert\") pod \"c76daa71-8efb-452c-806f-00c68893eb2b\" (UID: \"c76daa71-8efb-452c-806f-00c68893eb2b\") " Sep 08 18:28:29.913005 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:29.912868 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c76daa71-8efb-452c-806f-00c68893eb2b-service-ca" (OuterVolumeSpecName: "service-ca") pod "c76daa71-8efb-452c-806f-00c68893eb2b" (UID: "c76daa71-8efb-452c-806f-00c68893eb2b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:28:29.913005 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:29.912906 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c76daa71-8efb-452c-806f-00c68893eb2b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c76daa71-8efb-452c-806f-00c68893eb2b" (UID: "c76daa71-8efb-452c-806f-00c68893eb2b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:28:29.913005 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:29.912914 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c76daa71-8efb-452c-806f-00c68893eb2b-console-config" (OuterVolumeSpecName: "console-config") pod "c76daa71-8efb-452c-806f-00c68893eb2b" (UID: "c76daa71-8efb-452c-806f-00c68893eb2b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:28:29.913191 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:29.913163 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c76daa71-8efb-452c-806f-00c68893eb2b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c76daa71-8efb-452c-806f-00c68893eb2b" (UID: "c76daa71-8efb-452c-806f-00c68893eb2b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:28:29.913238 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:29.913174 2555 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c76daa71-8efb-452c-806f-00c68893eb2b-console-config\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:28:29.913238 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:29.913220 2555 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c76daa71-8efb-452c-806f-00c68893eb2b-oauth-serving-cert\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:28:29.913309 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:29.913237 2555 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c76daa71-8efb-452c-806f-00c68893eb2b-service-ca\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:28:29.914967 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:29.914932 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c76daa71-8efb-452c-806f-00c68893eb2b-kube-api-access-6kk4b" (OuterVolumeSpecName: "kube-api-access-6kk4b") pod "c76daa71-8efb-452c-806f-00c68893eb2b" (UID: "c76daa71-8efb-452c-806f-00c68893eb2b"). InnerVolumeSpecName "kube-api-access-6kk4b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:28:29.914967 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:29.914946 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c76daa71-8efb-452c-806f-00c68893eb2b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c76daa71-8efb-452c-806f-00c68893eb2b" (UID: "c76daa71-8efb-452c-806f-00c68893eb2b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 18:28:29.914967 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:29.914960 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c76daa71-8efb-452c-806f-00c68893eb2b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c76daa71-8efb-452c-806f-00c68893eb2b" (UID: "c76daa71-8efb-452c-806f-00c68893eb2b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 18:28:30.013726 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:30.013667 2555 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c76daa71-8efb-452c-806f-00c68893eb2b-trusted-ca-bundle\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:28:30.013726 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:30.013718 2555 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c76daa71-8efb-452c-806f-00c68893eb2b-console-serving-cert\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:28:30.013726 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:30.013731 2555 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c76daa71-8efb-452c-806f-00c68893eb2b-console-oauth-config\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:28:30.013955 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:30.013740 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6kk4b\" (UniqueName: \"kubernetes.io/projected/c76daa71-8efb-452c-806f-00c68893eb2b-kube-api-access-6kk4b\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:28:30.621602 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:30.621487 2555 generic.go:356] "Generic (PLEG): container finished" podID="c76daa71-8efb-452c-806f-00c68893eb2b" containerID="db4dfd41a383f723594a1034ccfb2e1fa6d3b8b343f8076394bcaf5530084850" exitCode=0 Sep 08 18:28:30.621602 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:30.621530 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76ff479d85-78l4q" Sep 08 18:28:30.621602 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:30.621556 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76ff479d85-78l4q" event={"ID":"c76daa71-8efb-452c-806f-00c68893eb2b","Type":"ContainerDied","Data":"db4dfd41a383f723594a1034ccfb2e1fa6d3b8b343f8076394bcaf5530084850"} Sep 08 18:28:30.622168 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:30.621618 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76ff479d85-78l4q" event={"ID":"c76daa71-8efb-452c-806f-00c68893eb2b","Type":"ContainerDied","Data":"936c4229d62b249c47fd94757de6007809a0bb42bdae4948ad12e66076f8c2f0"} Sep 08 18:28:30.622168 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:30.621639 2555 scope.go:117] "RemoveContainer" containerID="db4dfd41a383f723594a1034ccfb2e1fa6d3b8b343f8076394bcaf5530084850" Sep 08 18:28:30.629663 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:30.629644 2555 scope.go:117] "RemoveContainer" containerID="db4dfd41a383f723594a1034ccfb2e1fa6d3b8b343f8076394bcaf5530084850" Sep 08 18:28:30.629987 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:28:30.629971 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db4dfd41a383f723594a1034ccfb2e1fa6d3b8b343f8076394bcaf5530084850\": container with ID starting with db4dfd41a383f723594a1034ccfb2e1fa6d3b8b343f8076394bcaf5530084850 not found: ID does not exist" containerID="db4dfd41a383f723594a1034ccfb2e1fa6d3b8b343f8076394bcaf5530084850" Sep 08 18:28:30.630038 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:30.629995 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db4dfd41a383f723594a1034ccfb2e1fa6d3b8b343f8076394bcaf5530084850"} err="failed to get container status \"db4dfd41a383f723594a1034ccfb2e1fa6d3b8b343f8076394bcaf5530084850\": rpc error: code = NotFound desc = could not find container \"db4dfd41a383f723594a1034ccfb2e1fa6d3b8b343f8076394bcaf5530084850\": container with ID starting with db4dfd41a383f723594a1034ccfb2e1fa6d3b8b343f8076394bcaf5530084850 not found: ID does not exist" Sep 08 18:28:30.637195 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:30.637174 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76ff479d85-78l4q"] Sep 08 18:28:30.640897 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:30.640878 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-76ff479d85-78l4q"] Sep 08 18:28:32.322778 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:32.322747 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c76daa71-8efb-452c-806f-00c68893eb2b" path="/var/lib/kubelet/pods/c76daa71-8efb-452c-806f-00c68893eb2b/volumes" Sep 08 18:28:38.230507 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:38.230478 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9n28f_911c61b8-766b-4ae8-b046-784e22e85889/console-operator/1.log" Sep 08 18:28:38.234108 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:38.234084 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9n28f_911c61b8-766b-4ae8-b046-784e22e85889/console-operator/1.log" Sep 08 18:28:38.240537 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:38.240519 2555 kubelet.go:1648] "Image garbage collection succeeded" Sep 08 18:28:56.623046 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:56.623007 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:28:57.333900 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:57.333866 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5bpj"] Sep 08 18:28:57.334221 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:57.334210 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c76daa71-8efb-452c-806f-00c68893eb2b" containerName="console" Sep 08 18:28:57.334271 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:57.334223 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="c76daa71-8efb-452c-806f-00c68893eb2b" containerName="console" Sep 08 18:28:57.334310 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:57.334288 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="c76daa71-8efb-452c-806f-00c68893eb2b" containerName="console" Sep 08 18:28:57.338647 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:57.338630 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5bpj" Sep 08 18:28:57.340760 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:57.340740 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 08 18:28:57.340859 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:57.340765 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-4xn48\"" Sep 08 18:28:57.340859 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:57.340841 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 08 18:28:57.345655 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:57.345466 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5bpj"] Sep 08 18:28:57.434932 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:57.434901 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsgpk\" (UniqueName: \"kubernetes.io/projected/80a2fd44-4368-4868-872e-4f31bb50bbf3-kube-api-access-tsgpk\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5bpj\" (UID: \"80a2fd44-4368-4868-872e-4f31bb50bbf3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5bpj" Sep 08 18:28:57.435112 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:57.434984 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/80a2fd44-4368-4868-872e-4f31bb50bbf3-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5bpj\" (UID: \"80a2fd44-4368-4868-872e-4f31bb50bbf3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5bpj" Sep 08 18:28:57.435112 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:57.435026 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/80a2fd44-4368-4868-872e-4f31bb50bbf3-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5bpj\" (UID: \"80a2fd44-4368-4868-872e-4f31bb50bbf3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5bpj" Sep 08 18:28:57.535445 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:57.535420 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/80a2fd44-4368-4868-872e-4f31bb50bbf3-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5bpj\" (UID: \"80a2fd44-4368-4868-872e-4f31bb50bbf3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5bpj" Sep 08 18:28:57.535525 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:57.535467 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/80a2fd44-4368-4868-872e-4f31bb50bbf3-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5bpj\" (UID: \"80a2fd44-4368-4868-872e-4f31bb50bbf3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5bpj" Sep 08 18:28:57.535525 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:57.535499 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsgpk\" (UniqueName: \"kubernetes.io/projected/80a2fd44-4368-4868-872e-4f31bb50bbf3-kube-api-access-tsgpk\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5bpj\" (UID: \"80a2fd44-4368-4868-872e-4f31bb50bbf3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5bpj" Sep 08 18:28:57.535816 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:57.535794 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/80a2fd44-4368-4868-872e-4f31bb50bbf3-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5bpj\" (UID: \"80a2fd44-4368-4868-872e-4f31bb50bbf3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5bpj" Sep 08 18:28:57.535911 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:57.535839 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/80a2fd44-4368-4868-872e-4f31bb50bbf3-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5bpj\" (UID: \"80a2fd44-4368-4868-872e-4f31bb50bbf3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5bpj" Sep 08 18:28:57.544083 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:57.544065 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsgpk\" (UniqueName: \"kubernetes.io/projected/80a2fd44-4368-4868-872e-4f31bb50bbf3-kube-api-access-tsgpk\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5bpj\" (UID: \"80a2fd44-4368-4868-872e-4f31bb50bbf3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5bpj" Sep 08 18:28:57.649367 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:57.649271 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5bpj" Sep 08 18:28:57.917133 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:57.917083 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5bpj"] Sep 08 18:28:57.919390 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:28:57.919365 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80a2fd44_4368_4868_872e_4f31bb50bbf3.slice/crio-a193e591e6b00a35def6bc800d92e1678be0e88509abc0352388a79de9eb14ce WatchSource:0}: Error finding container a193e591e6b00a35def6bc800d92e1678be0e88509abc0352388a79de9eb14ce: Status 404 returned error can't find the container with id a193e591e6b00a35def6bc800d92e1678be0e88509abc0352388a79de9eb14ce Sep 08 18:28:57.921243 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:57.921229 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 08 18:28:58.068756 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:58.068719 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:28:58.739924 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:28:58.739886 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5bpj" event={"ID":"80a2fd44-4368-4868-872e-4f31bb50bbf3","Type":"ContainerStarted","Data":"a193e591e6b00a35def6bc800d92e1678be0e88509abc0352388a79de9eb14ce"} Sep 08 18:29:04.767291 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:04.767254 2555 generic.go:356] "Generic (PLEG): container finished" podID="80a2fd44-4368-4868-872e-4f31bb50bbf3" containerID="a2f59636e36bf420881b0a2aba78609a7f170916554853f8dc44e70919cf29bd" exitCode=0 Sep 08 18:29:04.767800 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:04.767323 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5bpj" event={"ID":"80a2fd44-4368-4868-872e-4f31bb50bbf3","Type":"ContainerDied","Data":"a2f59636e36bf420881b0a2aba78609a7f170916554853f8dc44e70919cf29bd"} Sep 08 18:29:07.783629 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:07.783590 2555 generic.go:356] "Generic (PLEG): container finished" podID="80a2fd44-4368-4868-872e-4f31bb50bbf3" containerID="b96cef73601eccb4bb7a858d7c888e739cd8114a25ed20005f46ba0faf5dce19" exitCode=0 Sep 08 18:29:07.784014 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:07.783668 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5bpj" event={"ID":"80a2fd44-4368-4868-872e-4f31bb50bbf3","Type":"ContainerDied","Data":"b96cef73601eccb4bb7a858d7c888e739cd8114a25ed20005f46ba0faf5dce19"} Sep 08 18:29:15.825997 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:15.825962 2555 generic.go:356] "Generic (PLEG): container finished" podID="80a2fd44-4368-4868-872e-4f31bb50bbf3" containerID="a765bbce28d3c9c423d2e2baf94bc2220ab4a28d6bb72f3c0a730194f6563526" exitCode=0 Sep 08 18:29:15.826381 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:15.826042 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5bpj" event={"ID":"80a2fd44-4368-4868-872e-4f31bb50bbf3","Type":"ContainerDied","Data":"a765bbce28d3c9c423d2e2baf94bc2220ab4a28d6bb72f3c0a730194f6563526"} Sep 08 18:29:16.997102 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:16.997069 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5bpj" Sep 08 18:29:17.108451 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:17.108401 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tsgpk\" (UniqueName: \"kubernetes.io/projected/80a2fd44-4368-4868-872e-4f31bb50bbf3-kube-api-access-tsgpk\") pod \"80a2fd44-4368-4868-872e-4f31bb50bbf3\" (UID: \"80a2fd44-4368-4868-872e-4f31bb50bbf3\") " Sep 08 18:29:17.108647 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:17.108504 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/80a2fd44-4368-4868-872e-4f31bb50bbf3-util\") pod \"80a2fd44-4368-4868-872e-4f31bb50bbf3\" (UID: \"80a2fd44-4368-4868-872e-4f31bb50bbf3\") " Sep 08 18:29:17.108647 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:17.108544 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/80a2fd44-4368-4868-872e-4f31bb50bbf3-bundle\") pod \"80a2fd44-4368-4868-872e-4f31bb50bbf3\" (UID: \"80a2fd44-4368-4868-872e-4f31bb50bbf3\") " Sep 08 18:29:17.109206 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:17.109184 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80a2fd44-4368-4868-872e-4f31bb50bbf3-bundle" (OuterVolumeSpecName: "bundle") pod "80a2fd44-4368-4868-872e-4f31bb50bbf3" (UID: "80a2fd44-4368-4868-872e-4f31bb50bbf3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:29:17.110859 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:17.110834 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80a2fd44-4368-4868-872e-4f31bb50bbf3-kube-api-access-tsgpk" (OuterVolumeSpecName: "kube-api-access-tsgpk") pod "80a2fd44-4368-4868-872e-4f31bb50bbf3" (UID: "80a2fd44-4368-4868-872e-4f31bb50bbf3"). InnerVolumeSpecName "kube-api-access-tsgpk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:29:17.112576 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:17.112556 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80a2fd44-4368-4868-872e-4f31bb50bbf3-util" (OuterVolumeSpecName: "util") pod "80a2fd44-4368-4868-872e-4f31bb50bbf3" (UID: "80a2fd44-4368-4868-872e-4f31bb50bbf3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:29:17.209956 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:17.209869 2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/80a2fd44-4368-4868-872e-4f31bb50bbf3-bundle\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:29:17.209956 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:17.209900 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tsgpk\" (UniqueName: \"kubernetes.io/projected/80a2fd44-4368-4868-872e-4f31bb50bbf3-kube-api-access-tsgpk\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:29:17.209956 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:17.209909 2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/80a2fd44-4368-4868-872e-4f31bb50bbf3-util\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:29:17.836249 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:17.836220 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5bpj" event={"ID":"80a2fd44-4368-4868-872e-4f31bb50bbf3","Type":"ContainerDied","Data":"a193e591e6b00a35def6bc800d92e1678be0e88509abc0352388a79de9eb14ce"} Sep 08 18:29:17.836249 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:17.836249 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z5bpj" Sep 08 18:29:17.836459 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:17.836263 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a193e591e6b00a35def6bc800d92e1678be0e88509abc0352388a79de9eb14ce" Sep 08 18:29:23.406229 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:23.406190 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:29:24.237050 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:24.237019 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:29:25.361849 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:25.361816 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6vnv5"] Sep 08 18:29:25.362214 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:25.362175 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80a2fd44-4368-4868-872e-4f31bb50bbf3" containerName="extract" Sep 08 18:29:25.362214 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:25.362187 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="80a2fd44-4368-4868-872e-4f31bb50bbf3" containerName="extract" Sep 08 18:29:25.362214 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:25.362202 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80a2fd44-4368-4868-872e-4f31bb50bbf3" containerName="util" Sep 08 18:29:25.362214 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:25.362211 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="80a2fd44-4368-4868-872e-4f31bb50bbf3" containerName="util" Sep 08 18:29:25.362339 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:25.362231 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80a2fd44-4368-4868-872e-4f31bb50bbf3" containerName="pull" Sep 08 18:29:25.362339 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:25.362236 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="80a2fd44-4368-4868-872e-4f31bb50bbf3" containerName="pull" Sep 08 18:29:25.362732 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:25.362667 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="80a2fd44-4368-4868-872e-4f31bb50bbf3" containerName="extract" Sep 08 18:29:25.369151 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:25.369129 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6vnv5" Sep 08 18:29:25.371279 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:25.371258 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Sep 08 18:29:25.371279 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:25.371269 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Sep 08 18:29:25.371470 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:25.371289 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-bf6w5\"" Sep 08 18:29:25.378201 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:25.378172 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6vnv5"] Sep 08 18:29:25.480258 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:25.480217 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l4fn\" (UniqueName: \"kubernetes.io/projected/1074644f-7976-477d-9b21-4288caf35996-kube-api-access-7l4fn\") pod \"cert-manager-operator-controller-manager-55c444bf86-6vnv5\" (UID: \"1074644f-7976-477d-9b21-4288caf35996\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6vnv5" Sep 08 18:29:25.480446 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:25.480301 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1074644f-7976-477d-9b21-4288caf35996-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6vnv5\" (UID: \"1074644f-7976-477d-9b21-4288caf35996\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6vnv5" Sep 08 18:29:25.581343 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:25.581306 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l4fn\" (UniqueName: \"kubernetes.io/projected/1074644f-7976-477d-9b21-4288caf35996-kube-api-access-7l4fn\") pod \"cert-manager-operator-controller-manager-55c444bf86-6vnv5\" (UID: \"1074644f-7976-477d-9b21-4288caf35996\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6vnv5" Sep 08 18:29:25.581343 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:25.581349 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1074644f-7976-477d-9b21-4288caf35996-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6vnv5\" (UID: \"1074644f-7976-477d-9b21-4288caf35996\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6vnv5" Sep 08 18:29:25.581734 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:25.581718 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1074644f-7976-477d-9b21-4288caf35996-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6vnv5\" (UID: \"1074644f-7976-477d-9b21-4288caf35996\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6vnv5" Sep 08 18:29:25.593197 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:25.593168 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l4fn\" (UniqueName: \"kubernetes.io/projected/1074644f-7976-477d-9b21-4288caf35996-kube-api-access-7l4fn\") pod \"cert-manager-operator-controller-manager-55c444bf86-6vnv5\" (UID: \"1074644f-7976-477d-9b21-4288caf35996\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6vnv5" Sep 08 18:29:25.678801 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:25.678724 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6vnv5" Sep 08 18:29:25.954006 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:25.953927 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6vnv5"] Sep 08 18:29:25.957330 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:29:25.957292 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1074644f_7976_477d_9b21_4288caf35996.slice/crio-db30da0105c1f981f354051591961570fbf12ace2da193207718b70b836a44bf WatchSource:0}: Error finding container db30da0105c1f981f354051591961570fbf12ace2da193207718b70b836a44bf: Status 404 returned error can't find the container with id db30da0105c1f981f354051591961570fbf12ace2da193207718b70b836a44bf Sep 08 18:29:26.876341 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:26.876299 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6vnv5" event={"ID":"1074644f-7976-477d-9b21-4288caf35996","Type":"ContainerStarted","Data":"db30da0105c1f981f354051591961570fbf12ace2da193207718b70b836a44bf"} Sep 08 18:29:26.999334 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:26.999298 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:29:27.099595 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:27.099555 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:29:28.784876 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:28.784844 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:29:28.884790 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:28.884756 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:29:28.886317 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:28.886290 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6vnv5" event={"ID":"1074644f-7976-477d-9b21-4288caf35996","Type":"ContainerStarted","Data":"9e823077fb11d7710b7212f88f5116b053e8a440cdae4c7755f7eeade96aeace"} Sep 08 18:29:28.906212 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:28.906165 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6vnv5" podStartSLOduration=2.019002141 podStartE2EDuration="3.906147166s" totalImagesPullingTime="1.887145025s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:29:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:29:28.904520224 +0000 UTC m=+351.110505633" watchObservedRunningTime="2026-09-08 18:29:28.906147166 +0000 UTC m=+351.112132564" Sep 08 18:29:29.247461 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:29.247370 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:29:29.418118 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:29.418089 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:29:29.856214 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:29.856186 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb42d8"] Sep 08 18:29:29.859443 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:29.859427 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb42d8" Sep 08 18:29:29.861555 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:29.861526 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 08 18:29:29.861673 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:29.861562 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 08 18:29:29.861673 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:29.861639 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-4xn48\"" Sep 08 18:29:29.867797 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:29.867776 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb42d8"] Sep 08 18:29:30.021039 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:30.021006 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-px7cr\" (UniqueName: \"kubernetes.io/projected/2f1182df-3a5c-4260-8bd7-9e12eff078ce-kube-api-access-px7cr\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb42d8\" (UID: \"2f1182df-3a5c-4260-8bd7-9e12eff078ce\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb42d8" Sep 08 18:29:30.021223 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:30.021117 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f1182df-3a5c-4260-8bd7-9e12eff078ce-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb42d8\" (UID: \"2f1182df-3a5c-4260-8bd7-9e12eff078ce\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb42d8" Sep 08 18:29:30.021223 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:30.021153 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f1182df-3a5c-4260-8bd7-9e12eff078ce-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb42d8\" (UID: \"2f1182df-3a5c-4260-8bd7-9e12eff078ce\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb42d8" Sep 08 18:29:30.122362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:30.122287 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f1182df-3a5c-4260-8bd7-9e12eff078ce-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb42d8\" (UID: \"2f1182df-3a5c-4260-8bd7-9e12eff078ce\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb42d8" Sep 08 18:29:30.122362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:30.122327 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f1182df-3a5c-4260-8bd7-9e12eff078ce-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb42d8\" (UID: \"2f1182df-3a5c-4260-8bd7-9e12eff078ce\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb42d8" Sep 08 18:29:30.122571 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:30.122380 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px7cr\" (UniqueName: \"kubernetes.io/projected/2f1182df-3a5c-4260-8bd7-9e12eff078ce-kube-api-access-px7cr\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb42d8\" (UID: \"2f1182df-3a5c-4260-8bd7-9e12eff078ce\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb42d8" Sep 08 18:29:30.122786 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:30.122765 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f1182df-3a5c-4260-8bd7-9e12eff078ce-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb42d8\" (UID: \"2f1182df-3a5c-4260-8bd7-9e12eff078ce\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb42d8" Sep 08 18:29:30.122864 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:30.122791 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f1182df-3a5c-4260-8bd7-9e12eff078ce-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb42d8\" (UID: \"2f1182df-3a5c-4260-8bd7-9e12eff078ce\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb42d8" Sep 08 18:29:30.135849 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:30.135812 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-px7cr\" (UniqueName: \"kubernetes.io/projected/2f1182df-3a5c-4260-8bd7-9e12eff078ce-kube-api-access-px7cr\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb42d8\" (UID: \"2f1182df-3a5c-4260-8bd7-9e12eff078ce\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb42d8" Sep 08 18:29:30.169651 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:30.169627 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb42d8" Sep 08 18:29:30.440365 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:30.440337 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb42d8"] Sep 08 18:29:30.442377 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:29:30.442352 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f1182df_3a5c_4260_8bd7_9e12eff078ce.slice/crio-6e66ff50122c661e5624766fe9a9f5ff796caeb135abdba80aa2fe5f5ee874ec WatchSource:0}: Error finding container 6e66ff50122c661e5624766fe9a9f5ff796caeb135abdba80aa2fe5f5ee874ec: Status 404 returned error can't find the container with id 6e66ff50122c661e5624766fe9a9f5ff796caeb135abdba80aa2fe5f5ee874ec Sep 08 18:29:30.896212 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:30.896177 2555 generic.go:356] "Generic (PLEG): container finished" podID="2f1182df-3a5c-4260-8bd7-9e12eff078ce" containerID="3009df828179599945d028f24db27ab7ec77c5847221ab0e2808d364a54c65e7" exitCode=0 Sep 08 18:29:30.896566 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:30.896263 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb42d8" event={"ID":"2f1182df-3a5c-4260-8bd7-9e12eff078ce","Type":"ContainerDied","Data":"3009df828179599945d028f24db27ab7ec77c5847221ab0e2808d364a54c65e7"} Sep 08 18:29:30.896566 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:30.896315 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb42d8" event={"ID":"2f1182df-3a5c-4260-8bd7-9e12eff078ce","Type":"ContainerStarted","Data":"6e66ff50122c661e5624766fe9a9f5ff796caeb135abdba80aa2fe5f5ee874ec"} Sep 08 18:29:31.185943 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:31.185834 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:29:31.404372 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:31.404332 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:29:31.794830 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:31.794796 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-hkpdh"] Sep 08 18:29:31.798269 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:31.798246 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hkpdh" Sep 08 18:29:31.800248 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:31.800231 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Sep 08 18:29:31.800789 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:31.800742 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Sep 08 18:29:31.800789 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:31.800747 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-hmrxv\"" Sep 08 18:29:31.807098 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:31.807058 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-hkpdh"] Sep 08 18:29:31.837389 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:31.837363 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/00de230c-e17a-45c8-a981-8a07df25dcb4-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-hkpdh\" (UID: \"00de230c-e17a-45c8-a981-8a07df25dcb4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hkpdh" Sep 08 18:29:31.837524 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:31.837400 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qk47\" (UniqueName: \"kubernetes.io/projected/00de230c-e17a-45c8-a981-8a07df25dcb4-kube-api-access-7qk47\") pod \"cert-manager-cainjector-559cdd8c87-hkpdh\" (UID: \"00de230c-e17a-45c8-a981-8a07df25dcb4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hkpdh" Sep 08 18:29:31.938370 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:31.938336 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qk47\" (UniqueName: \"kubernetes.io/projected/00de230c-e17a-45c8-a981-8a07df25dcb4-kube-api-access-7qk47\") pod \"cert-manager-cainjector-559cdd8c87-hkpdh\" (UID: \"00de230c-e17a-45c8-a981-8a07df25dcb4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hkpdh" Sep 08 18:29:31.938830 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:31.938471 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/00de230c-e17a-45c8-a981-8a07df25dcb4-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-hkpdh\" (UID: \"00de230c-e17a-45c8-a981-8a07df25dcb4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hkpdh" Sep 08 18:29:31.948314 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:31.948262 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/00de230c-e17a-45c8-a981-8a07df25dcb4-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-hkpdh\" (UID: \"00de230c-e17a-45c8-a981-8a07df25dcb4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hkpdh" Sep 08 18:29:31.948463 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:31.948326 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qk47\" (UniqueName: \"kubernetes.io/projected/00de230c-e17a-45c8-a981-8a07df25dcb4-kube-api-access-7qk47\") pod \"cert-manager-cainjector-559cdd8c87-hkpdh\" (UID: \"00de230c-e17a-45c8-a981-8a07df25dcb4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hkpdh" Sep 08 18:29:32.118580 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:32.118496 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hkpdh" Sep 08 18:29:32.397589 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:32.397561 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-hkpdh"] Sep 08 18:29:32.960952 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:29:32.960924 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod00de230c_e17a_45c8_a981_8a07df25dcb4.slice/crio-a53c1df9a33e8e1ad9671932de637c97e782d9e55a8738166e93ebf8745a4f76 WatchSource:0}: Error finding container a53c1df9a33e8e1ad9671932de637c97e782d9e55a8738166e93ebf8745a4f76: Status 404 returned error can't find the container with id a53c1df9a33e8e1ad9671932de637c97e782d9e55a8738166e93ebf8745a4f76 Sep 08 18:29:33.913435 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:33.913397 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hkpdh" event={"ID":"00de230c-e17a-45c8-a981-8a07df25dcb4","Type":"ContainerStarted","Data":"a53c1df9a33e8e1ad9671932de637c97e782d9e55a8738166e93ebf8745a4f76"} Sep 08 18:29:33.915540 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:33.915507 2555 generic.go:356] "Generic (PLEG): container finished" podID="2f1182df-3a5c-4260-8bd7-9e12eff078ce" containerID="71e0ee902b05fca07743d1e7b54044a5583557503d4fb93fdc54d3a8ec497a23" exitCode=0 Sep 08 18:29:33.915716 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:33.915557 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb42d8" event={"ID":"2f1182df-3a5c-4260-8bd7-9e12eff078ce","Type":"ContainerDied","Data":"71e0ee902b05fca07743d1e7b54044a5583557503d4fb93fdc54d3a8ec497a23"} Sep 08 18:29:34.004554 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:34.004524 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:29:34.106043 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:34.106006 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:29:34.923531 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:34.923498 2555 generic.go:356] "Generic (PLEG): container finished" podID="2f1182df-3a5c-4260-8bd7-9e12eff078ce" containerID="60f479ef29eb408146361c828d4e4528961990fd006628407065a909a213455b" exitCode=0 Sep 08 18:29:34.923778 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:34.923540 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb42d8" event={"ID":"2f1182df-3a5c-4260-8bd7-9e12eff078ce","Type":"ContainerDied","Data":"60f479ef29eb408146361c828d4e4528961990fd006628407065a909a213455b"} Sep 08 18:29:35.572352 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:35.572318 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-476g8"] Sep 08 18:29:35.575641 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:35.575626 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-476g8" Sep 08 18:29:35.578595 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:35.578549 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-cbs9r\"" Sep 08 18:29:35.616914 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:35.616888 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-476g8"] Sep 08 18:29:35.669331 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:35.669299 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjxxj\" (UniqueName: \"kubernetes.io/projected/7b413f4a-1485-4540-93da-6631d4695547-kube-api-access-sjxxj\") pod \"cert-manager-webhook-6486ffd997-476g8\" (UID: \"7b413f4a-1485-4540-93da-6631d4695547\") " pod="cert-manager/cert-manager-webhook-6486ffd997-476g8" Sep 08 18:29:35.669445 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:35.669352 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7b413f4a-1485-4540-93da-6631d4695547-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-476g8\" (UID: \"7b413f4a-1485-4540-93da-6631d4695547\") " pod="cert-manager/cert-manager-webhook-6486ffd997-476g8" Sep 08 18:29:35.769817 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:35.769776 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjxxj\" (UniqueName: \"kubernetes.io/projected/7b413f4a-1485-4540-93da-6631d4695547-kube-api-access-sjxxj\") pod \"cert-manager-webhook-6486ffd997-476g8\" (UID: \"7b413f4a-1485-4540-93da-6631d4695547\") " pod="cert-manager/cert-manager-webhook-6486ffd997-476g8" Sep 08 18:29:35.769986 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:35.769862 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7b413f4a-1485-4540-93da-6631d4695547-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-476g8\" (UID: \"7b413f4a-1485-4540-93da-6631d4695547\") " pod="cert-manager/cert-manager-webhook-6486ffd997-476g8" Sep 08 18:29:35.785425 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:35.785395 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7b413f4a-1485-4540-93da-6631d4695547-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-476g8\" (UID: \"7b413f4a-1485-4540-93da-6631d4695547\") " pod="cert-manager/cert-manager-webhook-6486ffd997-476g8" Sep 08 18:29:35.785562 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:35.785507 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjxxj\" (UniqueName: \"kubernetes.io/projected/7b413f4a-1485-4540-93da-6631d4695547-kube-api-access-sjxxj\") pod \"cert-manager-webhook-6486ffd997-476g8\" (UID: \"7b413f4a-1485-4540-93da-6631d4695547\") " pod="cert-manager/cert-manager-webhook-6486ffd997-476g8" Sep 08 18:29:35.884965 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:35.884878 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-476g8" Sep 08 18:29:35.931398 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:35.931361 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hkpdh" event={"ID":"00de230c-e17a-45c8-a981-8a07df25dcb4","Type":"ContainerStarted","Data":"4b4cfbecba999b4841c37a6456daf3d23728f6166da53945c43b2cfdf2808b9c"} Sep 08 18:29:35.954437 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:35.954373 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hkpdh" podStartSLOduration=2.327652446 podStartE2EDuration="4.954348272s" totalImagesPullingTime="2.626695826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:29:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:29:35.952234922 +0000 UTC m=+358.158220319" watchObservedRunningTime="2026-09-08 18:29:35.954348272 +0000 UTC m=+358.160333669" Sep 08 18:29:36.065340 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:36.065312 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb42d8" Sep 08 18:29:36.072744 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:36.072719 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-px7cr\" (UniqueName: \"kubernetes.io/projected/2f1182df-3a5c-4260-8bd7-9e12eff078ce-kube-api-access-px7cr\") pod \"2f1182df-3a5c-4260-8bd7-9e12eff078ce\" (UID: \"2f1182df-3a5c-4260-8bd7-9e12eff078ce\") " Sep 08 18:29:36.072867 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:36.072824 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f1182df-3a5c-4260-8bd7-9e12eff078ce-util\") pod \"2f1182df-3a5c-4260-8bd7-9e12eff078ce\" (UID: \"2f1182df-3a5c-4260-8bd7-9e12eff078ce\") " Sep 08 18:29:36.072867 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:36.072856 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f1182df-3a5c-4260-8bd7-9e12eff078ce-bundle\") pod \"2f1182df-3a5c-4260-8bd7-9e12eff078ce\" (UID: \"2f1182df-3a5c-4260-8bd7-9e12eff078ce\") " Sep 08 18:29:36.073229 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:36.073206 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f1182df-3a5c-4260-8bd7-9e12eff078ce-bundle" (OuterVolumeSpecName: "bundle") pod "2f1182df-3a5c-4260-8bd7-9e12eff078ce" (UID: "2f1182df-3a5c-4260-8bd7-9e12eff078ce"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:29:36.074633 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:36.074612 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f1182df-3a5c-4260-8bd7-9e12eff078ce-kube-api-access-px7cr" (OuterVolumeSpecName: "kube-api-access-px7cr") pod "2f1182df-3a5c-4260-8bd7-9e12eff078ce" (UID: "2f1182df-3a5c-4260-8bd7-9e12eff078ce"). InnerVolumeSpecName "kube-api-access-px7cr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:29:36.077007 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:36.076986 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f1182df-3a5c-4260-8bd7-9e12eff078ce-util" (OuterVolumeSpecName: "util") pod "2f1182df-3a5c-4260-8bd7-9e12eff078ce" (UID: "2f1182df-3a5c-4260-8bd7-9e12eff078ce"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:29:36.166965 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:36.166886 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-476g8"] Sep 08 18:29:36.169867 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:29:36.169845 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7b413f4a_1485_4540_93da_6631d4695547.slice/crio-dc45cdb3854b49fa7d8b3b7aec7f38dc7b94b116b42a588d97f1756ff3d23f12 WatchSource:0}: Error finding container dc45cdb3854b49fa7d8b3b7aec7f38dc7b94b116b42a588d97f1756ff3d23f12: Status 404 returned error can't find the container with id dc45cdb3854b49fa7d8b3b7aec7f38dc7b94b116b42a588d97f1756ff3d23f12 Sep 08 18:29:36.174179 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:36.174160 2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f1182df-3a5c-4260-8bd7-9e12eff078ce-util\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:29:36.174179 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:36.174179 2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f1182df-3a5c-4260-8bd7-9e12eff078ce-bundle\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:29:36.174296 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:36.174190 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-px7cr\" (UniqueName: \"kubernetes.io/projected/2f1182df-3a5c-4260-8bd7-9e12eff078ce-kube-api-access-px7cr\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:29:36.423394 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:36.423297 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:29:36.524400 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:36.524366 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:29:36.937219 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:36.937182 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-476g8" event={"ID":"7b413f4a-1485-4540-93da-6631d4695547","Type":"ContainerStarted","Data":"0b660b7100935f0f29a3737d834023da31b3ae53576fc48bf9330801dbfc7764"} Sep 08 18:29:36.937650 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:36.937235 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-476g8" Sep 08 18:29:36.937650 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:36.937249 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-476g8" event={"ID":"7b413f4a-1485-4540-93da-6631d4695547","Type":"ContainerStarted","Data":"dc45cdb3854b49fa7d8b3b7aec7f38dc7b94b116b42a588d97f1756ff3d23f12"} Sep 08 18:29:36.939105 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:36.939085 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb42d8" Sep 08 18:29:36.939214 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:36.939123 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb42d8" event={"ID":"2f1182df-3a5c-4260-8bd7-9e12eff078ce","Type":"ContainerDied","Data":"6e66ff50122c661e5624766fe9a9f5ff796caeb135abdba80aa2fe5f5ee874ec"} Sep 08 18:29:36.939214 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:36.939160 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6e66ff50122c661e5624766fe9a9f5ff796caeb135abdba80aa2fe5f5ee874ec" Sep 08 18:29:36.952499 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:36.952462 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-476g8" podStartSLOduration=1.952449822 podStartE2EDuration="1.952449822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:29:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:29:36.950881343 +0000 UTC m=+359.156866738" watchObservedRunningTime="2026-09-08 18:29:36.952449822 +0000 UTC m=+359.158435218" Sep 08 18:29:38.797545 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:38.797506 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:29:40.206997 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:40.206967 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:29:40.307861 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:40.307815 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:29:42.946202 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:42.946168 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-476g8" Sep 08 18:29:46.231625 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:46.231590 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:29:46.332436 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:46.332397 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:29:48.406355 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:48.406314 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqjblb"] Sep 08 18:29:48.406840 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:48.406812 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f1182df-3a5c-4260-8bd7-9e12eff078ce" containerName="util" Sep 08 18:29:48.406840 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:48.406829 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f1182df-3a5c-4260-8bd7-9e12eff078ce" containerName="util" Sep 08 18:29:48.406949 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:48.406859 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f1182df-3a5c-4260-8bd7-9e12eff078ce" containerName="pull" Sep 08 18:29:48.406949 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:48.406867 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f1182df-3a5c-4260-8bd7-9e12eff078ce" containerName="pull" Sep 08 18:29:48.406949 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:48.406895 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f1182df-3a5c-4260-8bd7-9e12eff078ce" containerName="extract" Sep 08 18:29:48.406949 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:48.406903 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f1182df-3a5c-4260-8bd7-9e12eff078ce" containerName="extract" Sep 08 18:29:48.407135 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:48.406983 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f1182df-3a5c-4260-8bd7-9e12eff078ce" containerName="extract" Sep 08 18:29:48.458618 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:48.458580 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqjblb"] Sep 08 18:29:48.458818 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:48.458729 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqjblb" Sep 08 18:29:48.461201 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:48.461157 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 08 18:29:48.461201 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:48.461171 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 08 18:29:48.461364 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:48.461326 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-4xn48\"" Sep 08 18:29:48.537529 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:48.537500 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:29:48.576875 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:48.576841 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d6b6968a-e89f-44e6-99b3-167588cfdac2-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqjblb\" (UID: \"d6b6968a-e89f-44e6-99b3-167588cfdac2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqjblb" Sep 08 18:29:48.577026 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:48.576894 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d6b6968a-e89f-44e6-99b3-167588cfdac2-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqjblb\" (UID: \"d6b6968a-e89f-44e6-99b3-167588cfdac2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqjblb" Sep 08 18:29:48.577026 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:48.576919 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bccf9\" (UniqueName: \"kubernetes.io/projected/d6b6968a-e89f-44e6-99b3-167588cfdac2-kube-api-access-bccf9\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqjblb\" (UID: \"d6b6968a-e89f-44e6-99b3-167588cfdac2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqjblb" Sep 08 18:29:48.677459 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:48.677367 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d6b6968a-e89f-44e6-99b3-167588cfdac2-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqjblb\" (UID: \"d6b6968a-e89f-44e6-99b3-167588cfdac2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqjblb" Sep 08 18:29:48.677459 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:48.677435 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d6b6968a-e89f-44e6-99b3-167588cfdac2-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqjblb\" (UID: \"d6b6968a-e89f-44e6-99b3-167588cfdac2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqjblb" Sep 08 18:29:48.677670 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:48.677467 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bccf9\" (UniqueName: \"kubernetes.io/projected/d6b6968a-e89f-44e6-99b3-167588cfdac2-kube-api-access-bccf9\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqjblb\" (UID: \"d6b6968a-e89f-44e6-99b3-167588cfdac2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqjblb" Sep 08 18:29:48.677879 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:48.677857 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d6b6968a-e89f-44e6-99b3-167588cfdac2-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqjblb\" (UID: \"d6b6968a-e89f-44e6-99b3-167588cfdac2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqjblb" Sep 08 18:29:48.677939 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:48.677912 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d6b6968a-e89f-44e6-99b3-167588cfdac2-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqjblb\" (UID: \"d6b6968a-e89f-44e6-99b3-167588cfdac2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqjblb" Sep 08 18:29:48.685352 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:48.685332 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bccf9\" (UniqueName: \"kubernetes.io/projected/d6b6968a-e89f-44e6-99b3-167588cfdac2-kube-api-access-bccf9\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqjblb\" (UID: \"d6b6968a-e89f-44e6-99b3-167588cfdac2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqjblb" Sep 08 18:29:48.768668 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:48.768638 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqjblb" Sep 08 18:29:49.057730 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:49.057705 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqjblb"] Sep 08 18:29:49.060182 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:29:49.060158 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6b6968a_e89f_44e6_99b3_167588cfdac2.slice/crio-2c4f85c7ceafe18097b7e3a5458fd0ccb2169130612d555da84e308c5387c8c9 WatchSource:0}: Error finding container 2c4f85c7ceafe18097b7e3a5458fd0ccb2169130612d555da84e308c5387c8c9: Status 404 returned error can't find the container with id 2c4f85c7ceafe18097b7e3a5458fd0ccb2169130612d555da84e308c5387c8c9 Sep 08 18:29:50.006581 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:50.006494 2555 generic.go:356] "Generic (PLEG): container finished" podID="d6b6968a-e89f-44e6-99b3-167588cfdac2" containerID="7e71a73b3e61a46dec568b90e29bdc271d3758e0662ea77ec496ba33edbf197c" exitCode=0 Sep 08 18:29:50.006581 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:50.006540 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqjblb" event={"ID":"d6b6968a-e89f-44e6-99b3-167588cfdac2","Type":"ContainerDied","Data":"7e71a73b3e61a46dec568b90e29bdc271d3758e0662ea77ec496ba33edbf197c"} Sep 08 18:29:50.007055 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:50.006622 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqjblb" event={"ID":"d6b6968a-e89f-44e6-99b3-167588cfdac2","Type":"ContainerStarted","Data":"2c4f85c7ceafe18097b7e3a5458fd0ccb2169130612d555da84e308c5387c8c9"} Sep 08 18:29:52.017070 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:52.017031 2555 generic.go:356] "Generic (PLEG): container finished" podID="d6b6968a-e89f-44e6-99b3-167588cfdac2" containerID="a9c69d3f774c9d8ff0f229057b9cd139ccd89f7d8c9fef937badf18afd78a25b" exitCode=0 Sep 08 18:29:52.017505 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:52.017074 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqjblb" event={"ID":"d6b6968a-e89f-44e6-99b3-167588cfdac2","Type":"ContainerDied","Data":"a9c69d3f774c9d8ff0f229057b9cd139ccd89f7d8c9fef937badf18afd78a25b"} Sep 08 18:29:53.023853 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:53.023819 2555 generic.go:356] "Generic (PLEG): container finished" podID="d6b6968a-e89f-44e6-99b3-167588cfdac2" containerID="4d0b8624790235447d6d05798000287c2230a6f685525bd2a4b92ccea7d2038a" exitCode=0 Sep 08 18:29:53.024283 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:53.023879 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqjblb" event={"ID":"d6b6968a-e89f-44e6-99b3-167588cfdac2","Type":"ContainerDied","Data":"4d0b8624790235447d6d05798000287c2230a6f685525bd2a4b92ccea7d2038a"} Sep 08 18:29:54.149612 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:54.149589 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqjblb" Sep 08 18:29:54.225253 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:54.225210 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d6b6968a-e89f-44e6-99b3-167588cfdac2-util\") pod \"d6b6968a-e89f-44e6-99b3-167588cfdac2\" (UID: \"d6b6968a-e89f-44e6-99b3-167588cfdac2\") " Sep 08 18:29:54.225253 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:54.225264 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bccf9\" (UniqueName: \"kubernetes.io/projected/d6b6968a-e89f-44e6-99b3-167588cfdac2-kube-api-access-bccf9\") pod \"d6b6968a-e89f-44e6-99b3-167588cfdac2\" (UID: \"d6b6968a-e89f-44e6-99b3-167588cfdac2\") " Sep 08 18:29:54.225458 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:54.225303 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d6b6968a-e89f-44e6-99b3-167588cfdac2-bundle\") pod \"d6b6968a-e89f-44e6-99b3-167588cfdac2\" (UID: \"d6b6968a-e89f-44e6-99b3-167588cfdac2\") " Sep 08 18:29:54.226029 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:54.226008 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6b6968a-e89f-44e6-99b3-167588cfdac2-bundle" (OuterVolumeSpecName: "bundle") pod "d6b6968a-e89f-44e6-99b3-167588cfdac2" (UID: "d6b6968a-e89f-44e6-99b3-167588cfdac2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:29:54.227430 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:54.227397 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6b6968a-e89f-44e6-99b3-167588cfdac2-kube-api-access-bccf9" (OuterVolumeSpecName: "kube-api-access-bccf9") pod "d6b6968a-e89f-44e6-99b3-167588cfdac2" (UID: "d6b6968a-e89f-44e6-99b3-167588cfdac2"). InnerVolumeSpecName "kube-api-access-bccf9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:29:54.230610 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:54.230591 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6b6968a-e89f-44e6-99b3-167588cfdac2-util" (OuterVolumeSpecName: "util") pod "d6b6968a-e89f-44e6-99b3-167588cfdac2" (UID: "d6b6968a-e89f-44e6-99b3-167588cfdac2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:29:54.325925 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:54.325902 2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d6b6968a-e89f-44e6-99b3-167588cfdac2-bundle\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:29:54.325925 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:54.325924 2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d6b6968a-e89f-44e6-99b3-167588cfdac2-util\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:29:54.326103 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:54.325933 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bccf9\" (UniqueName: \"kubernetes.io/projected/d6b6968a-e89f-44e6-99b3-167588cfdac2-kube-api-access-bccf9\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:29:55.035017 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:55.034988 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqjblb" event={"ID":"d6b6968a-e89f-44e6-99b3-167588cfdac2","Type":"ContainerDied","Data":"2c4f85c7ceafe18097b7e3a5458fd0ccb2169130612d555da84e308c5387c8c9"} Sep 08 18:29:55.035160 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:55.035020 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqjblb" Sep 08 18:29:55.035160 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:55.035035 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2c4f85c7ceafe18097b7e3a5458fd0ccb2169130612d555da84e308c5387c8c9" Sep 08 18:29:59.227655 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:59.227622 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mj7tp"] Sep 08 18:29:59.228096 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:59.227989 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6b6968a-e89f-44e6-99b3-167588cfdac2" containerName="extract" Sep 08 18:29:59.228096 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:59.228003 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6b6968a-e89f-44e6-99b3-167588cfdac2" containerName="extract" Sep 08 18:29:59.228096 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:59.228022 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6b6968a-e89f-44e6-99b3-167588cfdac2" containerName="util" Sep 08 18:29:59.228096 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:59.228027 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6b6968a-e89f-44e6-99b3-167588cfdac2" containerName="util" Sep 08 18:29:59.228096 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:59.228036 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6b6968a-e89f-44e6-99b3-167588cfdac2" containerName="pull" Sep 08 18:29:59.228096 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:59.228041 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6b6968a-e89f-44e6-99b3-167588cfdac2" containerName="pull" Sep 08 18:29:59.228096 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:59.228092 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6b6968a-e89f-44e6-99b3-167588cfdac2" containerName="extract" Sep 08 18:29:59.231551 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:59.231533 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mj7tp" Sep 08 18:29:59.233899 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:59.233876 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 08 18:29:59.234823 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:59.234808 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-4xn48\"" Sep 08 18:29:59.234949 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:59.234933 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 08 18:29:59.242096 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:59.242068 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mj7tp"] Sep 08 18:29:59.371705 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:59.371647 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llvbh\" (UniqueName: \"kubernetes.io/projected/c0b8bff5-1c64-4109-a304-323cc73d4086-kube-api-access-llvbh\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mj7tp\" (UID: \"c0b8bff5-1c64-4109-a304-323cc73d4086\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mj7tp" Sep 08 18:29:59.371874 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:59.371714 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0b8bff5-1c64-4109-a304-323cc73d4086-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mj7tp\" (UID: \"c0b8bff5-1c64-4109-a304-323cc73d4086\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mj7tp" Sep 08 18:29:59.371874 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:59.371769 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0b8bff5-1c64-4109-a304-323cc73d4086-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mj7tp\" (UID: \"c0b8bff5-1c64-4109-a304-323cc73d4086\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mj7tp" Sep 08 18:29:59.472917 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:59.472880 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0b8bff5-1c64-4109-a304-323cc73d4086-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mj7tp\" (UID: \"c0b8bff5-1c64-4109-a304-323cc73d4086\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mj7tp" Sep 08 18:29:59.473081 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:59.473003 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llvbh\" (UniqueName: \"kubernetes.io/projected/c0b8bff5-1c64-4109-a304-323cc73d4086-kube-api-access-llvbh\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mj7tp\" (UID: \"c0b8bff5-1c64-4109-a304-323cc73d4086\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mj7tp" Sep 08 18:29:59.473081 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:59.473041 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0b8bff5-1c64-4109-a304-323cc73d4086-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mj7tp\" (UID: \"c0b8bff5-1c64-4109-a304-323cc73d4086\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mj7tp" Sep 08 18:29:59.473359 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:59.473338 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0b8bff5-1c64-4109-a304-323cc73d4086-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mj7tp\" (UID: \"c0b8bff5-1c64-4109-a304-323cc73d4086\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mj7tp" Sep 08 18:29:59.473395 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:59.473344 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0b8bff5-1c64-4109-a304-323cc73d4086-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mj7tp\" (UID: \"c0b8bff5-1c64-4109-a304-323cc73d4086\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mj7tp" Sep 08 18:29:59.484552 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:59.484500 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llvbh\" (UniqueName: \"kubernetes.io/projected/c0b8bff5-1c64-4109-a304-323cc73d4086-kube-api-access-llvbh\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mj7tp\" (UID: \"c0b8bff5-1c64-4109-a304-323cc73d4086\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mj7tp" Sep 08 18:29:59.541731 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:59.541674 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mj7tp" Sep 08 18:29:59.811402 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:29:59.811377 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mj7tp"] Sep 08 18:29:59.813922 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:29:59.813895 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0b8bff5_1c64_4109_a304_323cc73d4086.slice/crio-ded129e4c4d4cec3013347ca979ab3513fd6e7d3a95ec7f1269ac9cf4a1e1efd WatchSource:0}: Error finding container ded129e4c4d4cec3013347ca979ab3513fd6e7d3a95ec7f1269ac9cf4a1e1efd: Status 404 returned error can't find the container with id ded129e4c4d4cec3013347ca979ab3513fd6e7d3a95ec7f1269ac9cf4a1e1efd Sep 08 18:30:00.011318 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:00.011274 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:00.058235 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:00.058201 2555 generic.go:356] "Generic (PLEG): container finished" podID="c0b8bff5-1c64-4109-a304-323cc73d4086" containerID="9d57251655f540fb0ee4db17e9e522ff863a15b8f00dab428e7653e8ab938cc2" exitCode=0 Sep 08 18:30:00.058389 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:00.058274 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mj7tp" event={"ID":"c0b8bff5-1c64-4109-a304-323cc73d4086","Type":"ContainerDied","Data":"9d57251655f540fb0ee4db17e9e522ff863a15b8f00dab428e7653e8ab938cc2"} Sep 08 18:30:00.058389 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:00.058320 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mj7tp" event={"ID":"c0b8bff5-1c64-4109-a304-323cc73d4086","Type":"ContainerStarted","Data":"ded129e4c4d4cec3013347ca979ab3513fd6e7d3a95ec7f1269ac9cf4a1e1efd"} Sep 08 18:30:00.092797 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:00.092731 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:00.374636 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:00.374554 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:00.650192 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:00.650118 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:00.755319 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:00.755289 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:00.787973 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:00.787951 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8d688d88f-l42d7"] Sep 08 18:30:00.792543 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:00.792522 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8d688d88f-l42d7" Sep 08 18:30:00.794846 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:00.794825 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 08 18:30:00.794846 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:00.794834 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 08 18:30:00.794993 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:00.794888 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-w9cwx\"" Sep 08 18:30:00.795051 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:00.795000 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 08 18:30:00.795138 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:00.795120 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 08 18:30:00.806485 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:00.806467 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8d688d88f-l42d7"] Sep 08 18:30:00.884656 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:00.884627 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2gjh\" (UniqueName: \"kubernetes.io/projected/684038e9-9cd1-457c-b852-d6eabc24e260-kube-api-access-x2gjh\") pod \"opendatahub-operator-controller-manager-8d688d88f-l42d7\" (UID: \"684038e9-9cd1-457c-b852-d6eabc24e260\") " pod="opendatahub/opendatahub-operator-controller-manager-8d688d88f-l42d7" Sep 08 18:30:00.884832 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:00.884707 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/684038e9-9cd1-457c-b852-d6eabc24e260-manifests\") pod \"opendatahub-operator-controller-manager-8d688d88f-l42d7\" (UID: \"684038e9-9cd1-457c-b852-d6eabc24e260\") " pod="opendatahub/opendatahub-operator-controller-manager-8d688d88f-l42d7" Sep 08 18:30:00.884832 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:00.884740 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/684038e9-9cd1-457c-b852-d6eabc24e260-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8d688d88f-l42d7\" (UID: \"684038e9-9cd1-457c-b852-d6eabc24e260\") " pod="opendatahub/opendatahub-operator-controller-manager-8d688d88f-l42d7" Sep 08 18:30:00.884926 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:00.884850 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/684038e9-9cd1-457c-b852-d6eabc24e260-webhook-cert\") pod \"opendatahub-operator-controller-manager-8d688d88f-l42d7\" (UID: \"684038e9-9cd1-457c-b852-d6eabc24e260\") " pod="opendatahub/opendatahub-operator-controller-manager-8d688d88f-l42d7" Sep 08 18:30:00.884926 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:00.884877 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/684038e9-9cd1-457c-b852-d6eabc24e260-tmp\") pod \"opendatahub-operator-controller-manager-8d688d88f-l42d7\" (UID: \"684038e9-9cd1-457c-b852-d6eabc24e260\") " pod="opendatahub/opendatahub-operator-controller-manager-8d688d88f-l42d7" Sep 08 18:30:00.965325 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:00.965246 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:00.985552 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:00.985514 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/684038e9-9cd1-457c-b852-d6eabc24e260-webhook-cert\") pod \"opendatahub-operator-controller-manager-8d688d88f-l42d7\" (UID: \"684038e9-9cd1-457c-b852-d6eabc24e260\") " pod="opendatahub/opendatahub-operator-controller-manager-8d688d88f-l42d7" Sep 08 18:30:00.985723 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:00.985555 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/684038e9-9cd1-457c-b852-d6eabc24e260-tmp\") pod \"opendatahub-operator-controller-manager-8d688d88f-l42d7\" (UID: \"684038e9-9cd1-457c-b852-d6eabc24e260\") " pod="opendatahub/opendatahub-operator-controller-manager-8d688d88f-l42d7" Sep 08 18:30:00.985723 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:00.985610 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2gjh\" (UniqueName: \"kubernetes.io/projected/684038e9-9cd1-457c-b852-d6eabc24e260-kube-api-access-x2gjh\") pod \"opendatahub-operator-controller-manager-8d688d88f-l42d7\" (UID: \"684038e9-9cd1-457c-b852-d6eabc24e260\") " pod="opendatahub/opendatahub-operator-controller-manager-8d688d88f-l42d7" Sep 08 18:30:00.985723 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:00.985653 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/684038e9-9cd1-457c-b852-d6eabc24e260-manifests\") pod \"opendatahub-operator-controller-manager-8d688d88f-l42d7\" (UID: \"684038e9-9cd1-457c-b852-d6eabc24e260\") " pod="opendatahub/opendatahub-operator-controller-manager-8d688d88f-l42d7" Sep 08 18:30:00.985723 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:00.985681 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/684038e9-9cd1-457c-b852-d6eabc24e260-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8d688d88f-l42d7\" (UID: \"684038e9-9cd1-457c-b852-d6eabc24e260\") " pod="opendatahub/opendatahub-operator-controller-manager-8d688d88f-l42d7" Sep 08 18:30:00.986126 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:00.986099 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/684038e9-9cd1-457c-b852-d6eabc24e260-manifests\") pod \"opendatahub-operator-controller-manager-8d688d88f-l42d7\" (UID: \"684038e9-9cd1-457c-b852-d6eabc24e260\") " pod="opendatahub/opendatahub-operator-controller-manager-8d688d88f-l42d7" Sep 08 18:30:00.987980 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:00.987954 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/684038e9-9cd1-457c-b852-d6eabc24e260-tmp\") pod \"opendatahub-operator-controller-manager-8d688d88f-l42d7\" (UID: \"684038e9-9cd1-457c-b852-d6eabc24e260\") " pod="opendatahub/opendatahub-operator-controller-manager-8d688d88f-l42d7" Sep 08 18:30:00.988283 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:00.988264 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/684038e9-9cd1-457c-b852-d6eabc24e260-webhook-cert\") pod \"opendatahub-operator-controller-manager-8d688d88f-l42d7\" (UID: \"684038e9-9cd1-457c-b852-d6eabc24e260\") " pod="opendatahub/opendatahub-operator-controller-manager-8d688d88f-l42d7" Sep 08 18:30:00.988335 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:00.988283 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/684038e9-9cd1-457c-b852-d6eabc24e260-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8d688d88f-l42d7\" (UID: \"684038e9-9cd1-457c-b852-d6eabc24e260\") " pod="opendatahub/opendatahub-operator-controller-manager-8d688d88f-l42d7" Sep 08 18:30:00.996875 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:00.996852 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2gjh\" (UniqueName: \"kubernetes.io/projected/684038e9-9cd1-457c-b852-d6eabc24e260-kube-api-access-x2gjh\") pod \"opendatahub-operator-controller-manager-8d688d88f-l42d7\" (UID: \"684038e9-9cd1-457c-b852-d6eabc24e260\") " pod="opendatahub/opendatahub-operator-controller-manager-8d688d88f-l42d7" Sep 08 18:30:01.058068 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:01.058040 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:01.064727 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:01.064704 2555 generic.go:356] "Generic (PLEG): container finished" podID="c0b8bff5-1c64-4109-a304-323cc73d4086" containerID="1fdf9c4e38932feaa870d14c9619aafc250c37a8762957045dabb0ee40b219e4" exitCode=0 Sep 08 18:30:01.064845 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:01.064761 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mj7tp" event={"ID":"c0b8bff5-1c64-4109-a304-323cc73d4086","Type":"ContainerDied","Data":"1fdf9c4e38932feaa870d14c9619aafc250c37a8762957045dabb0ee40b219e4"} Sep 08 18:30:01.104454 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:01.104434 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8d688d88f-l42d7" Sep 08 18:30:01.161498 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:01.161469 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:01.258645 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:01.258619 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:01.386034 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:01.386006 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:01.386998 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:01.386978 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8d688d88f-l42d7"] Sep 08 18:30:01.389553 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:30:01.389525 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod684038e9_9cd1_457c_b852_d6eabc24e260.slice/crio-32d3fd3c41a1bbe8bc17b5a6a0e0e854feb627777a79bacb9a8f5a25df257ab2 WatchSource:0}: Error finding container 32d3fd3c41a1bbe8bc17b5a6a0e0e854feb627777a79bacb9a8f5a25df257ab2: Status 404 returned error can't find the container with id 32d3fd3c41a1bbe8bc17b5a6a0e0e854feb627777a79bacb9a8f5a25df257ab2 Sep 08 18:30:02.071461 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:02.071426 2555 generic.go:356] "Generic (PLEG): container finished" podID="c0b8bff5-1c64-4109-a304-323cc73d4086" containerID="c93a76f93f2523d24834f5c205d0faac296773e4c7bb639a0093bb103f4f1255" exitCode=0 Sep 08 18:30:02.071624 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:02.071510 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mj7tp" event={"ID":"c0b8bff5-1c64-4109-a304-323cc73d4086","Type":"ContainerDied","Data":"c93a76f93f2523d24834f5c205d0faac296773e4c7bb639a0093bb103f4f1255"} Sep 08 18:30:02.072917 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:02.072892 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8d688d88f-l42d7" event={"ID":"684038e9-9cd1-457c-b852-d6eabc24e260","Type":"ContainerStarted","Data":"32d3fd3c41a1bbe8bc17b5a6a0e0e854feb627777a79bacb9a8f5a25df257ab2"} Sep 08 18:30:03.238818 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:03.238789 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mj7tp" Sep 08 18:30:03.306109 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:03.305918 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0b8bff5-1c64-4109-a304-323cc73d4086-bundle\") pod \"c0b8bff5-1c64-4109-a304-323cc73d4086\" (UID: \"c0b8bff5-1c64-4109-a304-323cc73d4086\") " Sep 08 18:30:03.306109 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:03.305966 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-llvbh\" (UniqueName: \"kubernetes.io/projected/c0b8bff5-1c64-4109-a304-323cc73d4086-kube-api-access-llvbh\") pod \"c0b8bff5-1c64-4109-a304-323cc73d4086\" (UID: \"c0b8bff5-1c64-4109-a304-323cc73d4086\") " Sep 08 18:30:03.306109 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:03.306039 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0b8bff5-1c64-4109-a304-323cc73d4086-util\") pod \"c0b8bff5-1c64-4109-a304-323cc73d4086\" (UID: \"c0b8bff5-1c64-4109-a304-323cc73d4086\") " Sep 08 18:30:03.306862 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:03.306836 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0b8bff5-1c64-4109-a304-323cc73d4086-bundle" (OuterVolumeSpecName: "bundle") pod "c0b8bff5-1c64-4109-a304-323cc73d4086" (UID: "c0b8bff5-1c64-4109-a304-323cc73d4086"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:30:03.310364 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:03.310324 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0b8bff5-1c64-4109-a304-323cc73d4086-kube-api-access-llvbh" (OuterVolumeSpecName: "kube-api-access-llvbh") pod "c0b8bff5-1c64-4109-a304-323cc73d4086" (UID: "c0b8bff5-1c64-4109-a304-323cc73d4086"). InnerVolumeSpecName "kube-api-access-llvbh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:30:03.314545 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:03.314502 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0b8bff5-1c64-4109-a304-323cc73d4086-util" (OuterVolumeSpecName: "util") pod "c0b8bff5-1c64-4109-a304-323cc73d4086" (UID: "c0b8bff5-1c64-4109-a304-323cc73d4086"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:30:03.407794 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:03.407547 2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0b8bff5-1c64-4109-a304-323cc73d4086-bundle\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:30:03.407794 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:03.407582 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-llvbh\" (UniqueName: \"kubernetes.io/projected/c0b8bff5-1c64-4109-a304-323cc73d4086-kube-api-access-llvbh\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:30:03.407794 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:03.407597 2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0b8bff5-1c64-4109-a304-323cc73d4086-util\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:30:04.087725 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:04.087674 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mj7tp" Sep 08 18:30:04.087725 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:04.087698 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0mj7tp" event={"ID":"c0b8bff5-1c64-4109-a304-323cc73d4086","Type":"ContainerDied","Data":"ded129e4c4d4cec3013347ca979ab3513fd6e7d3a95ec7f1269ac9cf4a1e1efd"} Sep 08 18:30:04.087949 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:04.087753 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ded129e4c4d4cec3013347ca979ab3513fd6e7d3a95ec7f1269ac9cf4a1e1efd" Sep 08 18:30:06.099291 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:06.099255 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8d688d88f-l42d7" event={"ID":"684038e9-9cd1-457c-b852-d6eabc24e260","Type":"ContainerStarted","Data":"b25acb0486bdead33a0b891b1a7043221a8050ce116ef785204f7f6760e7b69f"} Sep 08 18:30:07.105495 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:07.105463 2555 generic.go:356] "Generic (PLEG): container finished" podID="684038e9-9cd1-457c-b852-d6eabc24e260" containerID="b25acb0486bdead33a0b891b1a7043221a8050ce116ef785204f7f6760e7b69f" exitCode=0 Sep 08 18:30:07.105891 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:07.105552 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8d688d88f-l42d7" event={"ID":"684038e9-9cd1-457c-b852-d6eabc24e260","Type":"ContainerDied","Data":"b25acb0486bdead33a0b891b1a7043221a8050ce116ef785204f7f6760e7b69f"} Sep 08 18:30:08.112311 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:08.112277 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8d688d88f-l42d7" event={"ID":"684038e9-9cd1-457c-b852-d6eabc24e260","Type":"ContainerStarted","Data":"4e8f7287dc0f853a7f2324d4f9b7458973d358c1fbc5000b1e4ca5dd0e41dec0"} Sep 08 18:30:08.112705 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:08.112390 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-8d688d88f-l42d7" Sep 08 18:30:08.134066 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:08.134019 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-8d688d88f-l42d7" podStartSLOduration=2.500613968 podStartE2EDuration="8.133999475s" totalImagesPullingTime="4.576971096s" totalInitContainerRuntime="1.056414411s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:30:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:30:08.132303442 +0000 UTC m=+390.338288844" watchObservedRunningTime="2026-09-08 18:30:08.133999475 +0000 UTC m=+390.339984872" Sep 08 18:30:17.016424 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:17.016337 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7cbbd74665-bpntq"] Sep 08 18:30:17.017598 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:17.017570 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0b8bff5-1c64-4109-a304-323cc73d4086" containerName="pull" Sep 08 18:30:17.018736 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:17.018716 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0b8bff5-1c64-4109-a304-323cc73d4086" containerName="pull" Sep 08 18:30:17.018922 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:17.018891 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0b8bff5-1c64-4109-a304-323cc73d4086" containerName="util" Sep 08 18:30:17.018922 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:17.018909 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0b8bff5-1c64-4109-a304-323cc73d4086" containerName="util" Sep 08 18:30:17.019127 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:17.018944 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0b8bff5-1c64-4109-a304-323cc73d4086" containerName="extract" Sep 08 18:30:17.019127 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:17.018953 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0b8bff5-1c64-4109-a304-323cc73d4086" containerName="extract" Sep 08 18:30:17.019251 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:17.019136 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0b8bff5-1c64-4109-a304-323cc73d4086" containerName="extract" Sep 08 18:30:17.027925 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:17.027904 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cbbd74665-bpntq" Sep 08 18:30:17.040281 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:17.040256 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cbbd74665-bpntq"] Sep 08 18:30:17.092036 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:17.091998 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:17.130661 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:17.130633 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/117aa701-7af8-4fc8-9477-74e60ae6e463-console-serving-cert\") pod \"console-7cbbd74665-bpntq\" (UID: \"117aa701-7af8-4fc8-9477-74e60ae6e463\") " pod="openshift-console/console-7cbbd74665-bpntq" Sep 08 18:30:17.130661 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:17.130665 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/117aa701-7af8-4fc8-9477-74e60ae6e463-service-ca\") pod \"console-7cbbd74665-bpntq\" (UID: \"117aa701-7af8-4fc8-9477-74e60ae6e463\") " pod="openshift-console/console-7cbbd74665-bpntq" Sep 08 18:30:17.130859 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:17.130711 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/117aa701-7af8-4fc8-9477-74e60ae6e463-console-oauth-config\") pod \"console-7cbbd74665-bpntq\" (UID: \"117aa701-7af8-4fc8-9477-74e60ae6e463\") " pod="openshift-console/console-7cbbd74665-bpntq" Sep 08 18:30:17.130859 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:17.130734 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/117aa701-7af8-4fc8-9477-74e60ae6e463-trusted-ca-bundle\") pod \"console-7cbbd74665-bpntq\" (UID: \"117aa701-7af8-4fc8-9477-74e60ae6e463\") " pod="openshift-console/console-7cbbd74665-bpntq" Sep 08 18:30:17.130859 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:17.130752 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/117aa701-7af8-4fc8-9477-74e60ae6e463-oauth-serving-cert\") pod \"console-7cbbd74665-bpntq\" (UID: \"117aa701-7af8-4fc8-9477-74e60ae6e463\") " pod="openshift-console/console-7cbbd74665-bpntq" Sep 08 18:30:17.130859 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:17.130825 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw5kj\" (UniqueName: \"kubernetes.io/projected/117aa701-7af8-4fc8-9477-74e60ae6e463-kube-api-access-nw5kj\") pod \"console-7cbbd74665-bpntq\" (UID: \"117aa701-7af8-4fc8-9477-74e60ae6e463\") " pod="openshift-console/console-7cbbd74665-bpntq" Sep 08 18:30:17.130995 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:17.130925 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/117aa701-7af8-4fc8-9477-74e60ae6e463-console-config\") pod \"console-7cbbd74665-bpntq\" (UID: \"117aa701-7af8-4fc8-9477-74e60ae6e463\") " pod="openshift-console/console-7cbbd74665-bpntq" Sep 08 18:30:17.192424 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:17.192396 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:17.232131 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:17.232102 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/117aa701-7af8-4fc8-9477-74e60ae6e463-console-config\") pod \"console-7cbbd74665-bpntq\" (UID: \"117aa701-7af8-4fc8-9477-74e60ae6e463\") " pod="openshift-console/console-7cbbd74665-bpntq" Sep 08 18:30:17.232278 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:17.232146 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/117aa701-7af8-4fc8-9477-74e60ae6e463-console-serving-cert\") pod \"console-7cbbd74665-bpntq\" (UID: \"117aa701-7af8-4fc8-9477-74e60ae6e463\") " pod="openshift-console/console-7cbbd74665-bpntq" Sep 08 18:30:17.232278 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:17.232170 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/117aa701-7af8-4fc8-9477-74e60ae6e463-service-ca\") pod \"console-7cbbd74665-bpntq\" (UID: \"117aa701-7af8-4fc8-9477-74e60ae6e463\") " pod="openshift-console/console-7cbbd74665-bpntq" Sep 08 18:30:17.232278 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:17.232197 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/117aa701-7af8-4fc8-9477-74e60ae6e463-console-oauth-config\") pod \"console-7cbbd74665-bpntq\" (UID: \"117aa701-7af8-4fc8-9477-74e60ae6e463\") " pod="openshift-console/console-7cbbd74665-bpntq" Sep 08 18:30:17.232278 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:17.232219 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/117aa701-7af8-4fc8-9477-74e60ae6e463-trusted-ca-bundle\") pod \"console-7cbbd74665-bpntq\" (UID: \"117aa701-7af8-4fc8-9477-74e60ae6e463\") " pod="openshift-console/console-7cbbd74665-bpntq" Sep 08 18:30:17.232278 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:17.232241 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/117aa701-7af8-4fc8-9477-74e60ae6e463-oauth-serving-cert\") pod \"console-7cbbd74665-bpntq\" (UID: \"117aa701-7af8-4fc8-9477-74e60ae6e463\") " pod="openshift-console/console-7cbbd74665-bpntq" Sep 08 18:30:17.232278 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:17.232271 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw5kj\" (UniqueName: \"kubernetes.io/projected/117aa701-7af8-4fc8-9477-74e60ae6e463-kube-api-access-nw5kj\") pod \"console-7cbbd74665-bpntq\" (UID: \"117aa701-7af8-4fc8-9477-74e60ae6e463\") " pod="openshift-console/console-7cbbd74665-bpntq" Sep 08 18:30:17.232971 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:17.232933 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/117aa701-7af8-4fc8-9477-74e60ae6e463-console-config\") pod \"console-7cbbd74665-bpntq\" (UID: \"117aa701-7af8-4fc8-9477-74e60ae6e463\") " pod="openshift-console/console-7cbbd74665-bpntq" Sep 08 18:30:17.233125 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:17.232997 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/117aa701-7af8-4fc8-9477-74e60ae6e463-oauth-serving-cert\") pod \"console-7cbbd74665-bpntq\" (UID: \"117aa701-7af8-4fc8-9477-74e60ae6e463\") " pod="openshift-console/console-7cbbd74665-bpntq" Sep 08 18:30:17.233202 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:17.233181 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/117aa701-7af8-4fc8-9477-74e60ae6e463-service-ca\") pod \"console-7cbbd74665-bpntq\" (UID: \"117aa701-7af8-4fc8-9477-74e60ae6e463\") " pod="openshift-console/console-7cbbd74665-bpntq" Sep 08 18:30:17.233315 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:17.233294 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/117aa701-7af8-4fc8-9477-74e60ae6e463-trusted-ca-bundle\") pod \"console-7cbbd74665-bpntq\" (UID: \"117aa701-7af8-4fc8-9477-74e60ae6e463\") " pod="openshift-console/console-7cbbd74665-bpntq" Sep 08 18:30:17.234913 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:17.234891 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/117aa701-7af8-4fc8-9477-74e60ae6e463-console-oauth-config\") pod \"console-7cbbd74665-bpntq\" (UID: \"117aa701-7af8-4fc8-9477-74e60ae6e463\") " pod="openshift-console/console-7cbbd74665-bpntq" Sep 08 18:30:17.234980 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:17.234967 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/117aa701-7af8-4fc8-9477-74e60ae6e463-console-serving-cert\") pod \"console-7cbbd74665-bpntq\" (UID: \"117aa701-7af8-4fc8-9477-74e60ae6e463\") " pod="openshift-console/console-7cbbd74665-bpntq" Sep 08 18:30:17.240844 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:17.240822 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw5kj\" (UniqueName: \"kubernetes.io/projected/117aa701-7af8-4fc8-9477-74e60ae6e463-kube-api-access-nw5kj\") pod \"console-7cbbd74665-bpntq\" (UID: \"117aa701-7af8-4fc8-9477-74e60ae6e463\") " pod="openshift-console/console-7cbbd74665-bpntq" Sep 08 18:30:17.337621 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:17.337591 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cbbd74665-bpntq" Sep 08 18:30:17.629041 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:17.629012 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cbbd74665-bpntq"] Sep 08 18:30:17.630473 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:30:17.630446 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod117aa701_7af8_4fc8_9477_74e60ae6e463.slice/crio-9c24a89f3c09283c98c32873093140bd25762120db9ea7301bb6bd37a5eb8434 WatchSource:0}: Error finding container 9c24a89f3c09283c98c32873093140bd25762120db9ea7301bb6bd37a5eb8434: Status 404 returned error can't find the container with id 9c24a89f3c09283c98c32873093140bd25762120db9ea7301bb6bd37a5eb8434 Sep 08 18:30:18.161349 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:18.161311 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cbbd74665-bpntq" event={"ID":"117aa701-7af8-4fc8-9477-74e60ae6e463","Type":"ContainerStarted","Data":"676119064cbb193f9eade0760e75bf5acb3f6ee342fb552ec85777d4713e2c63"} Sep 08 18:30:18.161851 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:18.161367 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cbbd74665-bpntq" event={"ID":"117aa701-7af8-4fc8-9477-74e60ae6e463","Type":"ContainerStarted","Data":"9c24a89f3c09283c98c32873093140bd25762120db9ea7301bb6bd37a5eb8434"} Sep 08 18:30:18.180857 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:18.180809 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7cbbd74665-bpntq" podStartSLOduration=2.180789537 podStartE2EDuration="2.180789537s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:30:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:30:18.179350479 +0000 UTC m=+400.385335875" watchObservedRunningTime="2026-09-08 18:30:18.180789537 +0000 UTC m=+400.386774935" Sep 08 18:30:18.564170 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:18.564119 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:18.665859 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:18.665822 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:19.120001 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:19.119966 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-8d688d88f-l42d7" Sep 08 18:30:20.671418 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:20.671390 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6b68db98f8-5t66l"] Sep 08 18:30:20.674949 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:20.674931 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6b68db98f8-5t66l" Sep 08 18:30:20.677215 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:20.677193 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Sep 08 18:30:20.677962 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:20.677942 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Sep 08 18:30:20.678069 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:20.677977 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Sep 08 18:30:20.678131 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:20.678080 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-s8snx\"" Sep 08 18:30:20.678131 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:20.678079 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Sep 08 18:30:20.678131 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:20.678104 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Sep 08 18:30:20.696092 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:20.696070 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6b68db98f8-5t66l"] Sep 08 18:30:20.768025 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:20.767997 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8d6c1b59-4982-477e-b4e6-878226168e14-manager-config\") pod \"lws-controller-manager-6b68db98f8-5t66l\" (UID: \"8d6c1b59-4982-477e-b4e6-878226168e14\") " pod="openshift-lws-operator/lws-controller-manager-6b68db98f8-5t66l" Sep 08 18:30:20.768178 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:20.768028 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8d6c1b59-4982-477e-b4e6-878226168e14-metrics-cert\") pod \"lws-controller-manager-6b68db98f8-5t66l\" (UID: \"8d6c1b59-4982-477e-b4e6-878226168e14\") " pod="openshift-lws-operator/lws-controller-manager-6b68db98f8-5t66l" Sep 08 18:30:20.768178 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:20.768064 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8d6c1b59-4982-477e-b4e6-878226168e14-cert\") pod \"lws-controller-manager-6b68db98f8-5t66l\" (UID: \"8d6c1b59-4982-477e-b4e6-878226168e14\") " pod="openshift-lws-operator/lws-controller-manager-6b68db98f8-5t66l" Sep 08 18:30:20.768178 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:20.768079 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zdb9\" (UniqueName: \"kubernetes.io/projected/8d6c1b59-4982-477e-b4e6-878226168e14-kube-api-access-9zdb9\") pod \"lws-controller-manager-6b68db98f8-5t66l\" (UID: \"8d6c1b59-4982-477e-b4e6-878226168e14\") " pod="openshift-lws-operator/lws-controller-manager-6b68db98f8-5t66l" Sep 08 18:30:20.869048 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:20.869010 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8d6c1b59-4982-477e-b4e6-878226168e14-manager-config\") pod \"lws-controller-manager-6b68db98f8-5t66l\" (UID: \"8d6c1b59-4982-477e-b4e6-878226168e14\") " pod="openshift-lws-operator/lws-controller-manager-6b68db98f8-5t66l" Sep 08 18:30:20.869048 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:20.869051 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8d6c1b59-4982-477e-b4e6-878226168e14-metrics-cert\") pod \"lws-controller-manager-6b68db98f8-5t66l\" (UID: \"8d6c1b59-4982-477e-b4e6-878226168e14\") " pod="openshift-lws-operator/lws-controller-manager-6b68db98f8-5t66l" Sep 08 18:30:20.869274 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:20.869090 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8d6c1b59-4982-477e-b4e6-878226168e14-cert\") pod \"lws-controller-manager-6b68db98f8-5t66l\" (UID: \"8d6c1b59-4982-477e-b4e6-878226168e14\") " pod="openshift-lws-operator/lws-controller-manager-6b68db98f8-5t66l" Sep 08 18:30:20.869274 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:20.869112 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zdb9\" (UniqueName: \"kubernetes.io/projected/8d6c1b59-4982-477e-b4e6-878226168e14-kube-api-access-9zdb9\") pod \"lws-controller-manager-6b68db98f8-5t66l\" (UID: \"8d6c1b59-4982-477e-b4e6-878226168e14\") " pod="openshift-lws-operator/lws-controller-manager-6b68db98f8-5t66l" Sep 08 18:30:20.869821 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:20.869739 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8d6c1b59-4982-477e-b4e6-878226168e14-manager-config\") pod \"lws-controller-manager-6b68db98f8-5t66l\" (UID: \"8d6c1b59-4982-477e-b4e6-878226168e14\") " pod="openshift-lws-operator/lws-controller-manager-6b68db98f8-5t66l" Sep 08 18:30:20.871638 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:20.871618 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8d6c1b59-4982-477e-b4e6-878226168e14-metrics-cert\") pod \"lws-controller-manager-6b68db98f8-5t66l\" (UID: \"8d6c1b59-4982-477e-b4e6-878226168e14\") " pod="openshift-lws-operator/lws-controller-manager-6b68db98f8-5t66l" Sep 08 18:30:20.871749 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:20.871715 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8d6c1b59-4982-477e-b4e6-878226168e14-cert\") pod \"lws-controller-manager-6b68db98f8-5t66l\" (UID: \"8d6c1b59-4982-477e-b4e6-878226168e14\") " pod="openshift-lws-operator/lws-controller-manager-6b68db98f8-5t66l" Sep 08 18:30:20.889761 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:20.889741 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zdb9\" (UniqueName: \"kubernetes.io/projected/8d6c1b59-4982-477e-b4e6-878226168e14-kube-api-access-9zdb9\") pod \"lws-controller-manager-6b68db98f8-5t66l\" (UID: \"8d6c1b59-4982-477e-b4e6-878226168e14\") " pod="openshift-lws-operator/lws-controller-manager-6b68db98f8-5t66l" Sep 08 18:30:20.984780 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:20.984700 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6b68db98f8-5t66l" Sep 08 18:30:21.008211 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:21.008184 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:21.140362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:21.140334 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:21.272254 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:21.272230 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6b68db98f8-5t66l"] Sep 08 18:30:21.273973 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:30:21.273946 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d6c1b59_4982_477e_b4e6_878226168e14.slice/crio-aeba363eeab225f484fae716fc4e75f1b9535f85bfac240d6d0e4469eae80b47 WatchSource:0}: Error finding container aeba363eeab225f484fae716fc4e75f1b9535f85bfac240d6d0e4469eae80b47: Status 404 returned error can't find the container with id aeba363eeab225f484fae716fc4e75f1b9535f85bfac240d6d0e4469eae80b47 Sep 08 18:30:22.185992 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:22.185940 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6b68db98f8-5t66l" event={"ID":"8d6c1b59-4982-477e-b4e6-878226168e14","Type":"ContainerStarted","Data":"aeba363eeab225f484fae716fc4e75f1b9535f85bfac240d6d0e4469eae80b47"} Sep 08 18:30:23.191944 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:23.191911 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6b68db98f8-5t66l" event={"ID":"8d6c1b59-4982-477e-b4e6-878226168e14","Type":"ContainerStarted","Data":"a457e388f181972a78f670122ec6ea93ca94240c7bf82a17971b9904d7443d69"} Sep 08 18:30:23.192304 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:23.191973 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6b68db98f8-5t66l" Sep 08 18:30:23.210256 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:23.210208 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6b68db98f8-5t66l" podStartSLOduration=1.9688775029999999 podStartE2EDuration="3.21019236s" totalImagesPullingTime="1.241314857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:30:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:30:23.208207928 +0000 UTC m=+405.414193326" watchObservedRunningTime="2026-09-08 18:30:23.21019236 +0000 UTC m=+405.416177757" Sep 08 18:30:27.338549 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:27.338511 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7cbbd74665-bpntq" Sep 08 18:30:27.338986 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:27.338597 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7cbbd74665-bpntq" Sep 08 18:30:27.343063 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:27.343046 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7cbbd74665-bpntq" Sep 08 18:30:28.225560 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:28.225517 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7cbbd74665-bpntq" Sep 08 18:30:28.270464 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:28.270433 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85cd9df9bd-pgq8c"] Sep 08 18:30:28.377711 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:28.377638 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:28.479924 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:28.479838 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:28.535579 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:28.535544 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:31.745556 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:31.745515 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:31.779426 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:31.779399 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6d985cc74d-b2kg9"] Sep 08 18:30:31.782620 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:31.782603 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6d985cc74d-b2kg9" Sep 08 18:30:31.784889 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:31.784833 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-hsw52\"" Sep 08 18:30:31.785060 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:31.784904 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Sep 08 18:30:31.785060 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:31.784918 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Sep 08 18:30:31.794200 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:31.794170 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6d985cc74d-b2kg9"] Sep 08 18:30:31.864917 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:31.864884 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c1b6bd1-08a0-4fce-be8c-648216b13ebe-tls-certs\") pod \"kube-auth-proxy-6d985cc74d-b2kg9\" (UID: \"7c1b6bd1-08a0-4fce-be8c-648216b13ebe\") " pod="openshift-ingress/kube-auth-proxy-6d985cc74d-b2kg9" Sep 08 18:30:31.864917 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:31.864918 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c1b6bd1-08a0-4fce-be8c-648216b13ebe-tmp\") pod \"kube-auth-proxy-6d985cc74d-b2kg9\" (UID: \"7c1b6bd1-08a0-4fce-be8c-648216b13ebe\") " pod="openshift-ingress/kube-auth-proxy-6d985cc74d-b2kg9" Sep 08 18:30:31.865125 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:31.864935 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmr97\" (UniqueName: \"kubernetes.io/projected/7c1b6bd1-08a0-4fce-be8c-648216b13ebe-kube-api-access-xmr97\") pod \"kube-auth-proxy-6d985cc74d-b2kg9\" (UID: \"7c1b6bd1-08a0-4fce-be8c-648216b13ebe\") " pod="openshift-ingress/kube-auth-proxy-6d985cc74d-b2kg9" Sep 08 18:30:31.966342 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:31.966313 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c1b6bd1-08a0-4fce-be8c-648216b13ebe-tls-certs\") pod \"kube-auth-proxy-6d985cc74d-b2kg9\" (UID: \"7c1b6bd1-08a0-4fce-be8c-648216b13ebe\") " pod="openshift-ingress/kube-auth-proxy-6d985cc74d-b2kg9" Sep 08 18:30:31.966342 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:31.966347 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c1b6bd1-08a0-4fce-be8c-648216b13ebe-tmp\") pod \"kube-auth-proxy-6d985cc74d-b2kg9\" (UID: \"7c1b6bd1-08a0-4fce-be8c-648216b13ebe\") " pod="openshift-ingress/kube-auth-proxy-6d985cc74d-b2kg9" Sep 08 18:30:31.966585 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:31.966368 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmr97\" (UniqueName: \"kubernetes.io/projected/7c1b6bd1-08a0-4fce-be8c-648216b13ebe-kube-api-access-xmr97\") pod \"kube-auth-proxy-6d985cc74d-b2kg9\" (UID: \"7c1b6bd1-08a0-4fce-be8c-648216b13ebe\") " pod="openshift-ingress/kube-auth-proxy-6d985cc74d-b2kg9" Sep 08 18:30:31.966585 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:30:31.966481 2555 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Sep 08 18:30:31.966714 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:30:31.966609 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c1b6bd1-08a0-4fce-be8c-648216b13ebe-tls-certs podName:7c1b6bd1-08a0-4fce-be8c-648216b13ebe nodeName:}" failed. No retries permitted until 2026-09-08 18:30:32.466588386 +0000 UTC m=+414.672573764 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/7c1b6bd1-08a0-4fce-be8c-648216b13ebe-tls-certs") pod "kube-auth-proxy-6d985cc74d-b2kg9" (UID: "7c1b6bd1-08a0-4fce-be8c-648216b13ebe") : secret "kube-auth-proxy-tls" not found Sep 08 18:30:31.969045 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:31.969018 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c1b6bd1-08a0-4fce-be8c-648216b13ebe-tmp\") pod \"kube-auth-proxy-6d985cc74d-b2kg9\" (UID: \"7c1b6bd1-08a0-4fce-be8c-648216b13ebe\") " pod="openshift-ingress/kube-auth-proxy-6d985cc74d-b2kg9" Sep 08 18:30:31.982191 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:31.982160 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmr97\" (UniqueName: \"kubernetes.io/projected/7c1b6bd1-08a0-4fce-be8c-648216b13ebe-kube-api-access-xmr97\") pod \"kube-auth-proxy-6d985cc74d-b2kg9\" (UID: \"7c1b6bd1-08a0-4fce-be8c-648216b13ebe\") " pod="openshift-ingress/kube-auth-proxy-6d985cc74d-b2kg9" Sep 08 18:30:31.994104 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:31.994070 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:32.092016 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:32.091985 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:32.192515 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:32.192485 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:32.293799 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:32.293766 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:32.472360 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:32.472270 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c1b6bd1-08a0-4fce-be8c-648216b13ebe-tls-certs\") pod \"kube-auth-proxy-6d985cc74d-b2kg9\" (UID: \"7c1b6bd1-08a0-4fce-be8c-648216b13ebe\") " pod="openshift-ingress/kube-auth-proxy-6d985cc74d-b2kg9" Sep 08 18:30:32.475042 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:32.475016 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c1b6bd1-08a0-4fce-be8c-648216b13ebe-tls-certs\") pod \"kube-auth-proxy-6d985cc74d-b2kg9\" (UID: \"7c1b6bd1-08a0-4fce-be8c-648216b13ebe\") " pod="openshift-ingress/kube-auth-proxy-6d985cc74d-b2kg9" Sep 08 18:30:32.588701 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:32.588648 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:32.691481 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:32.691450 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6d985cc74d-b2kg9" Sep 08 18:30:32.973287 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:32.973265 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6d985cc74d-b2kg9"] Sep 08 18:30:32.974661 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:30:32.974636 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c1b6bd1_08a0_4fce_be8c_648216b13ebe.slice/crio-dc5316050aea8ddff8b8d2b9fc17f1896e3a1a6d37edbc5bbb14fd8637106fdb WatchSource:0}: Error finding container dc5316050aea8ddff8b8d2b9fc17f1896e3a1a6d37edbc5bbb14fd8637106fdb: Status 404 returned error can't find the container with id dc5316050aea8ddff8b8d2b9fc17f1896e3a1a6d37edbc5bbb14fd8637106fdb Sep 08 18:30:33.113857 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:33.113829 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:33.191222 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:33.191188 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:33.248705 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:33.248646 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6d985cc74d-b2kg9" event={"ID":"7c1b6bd1-08a0-4fce-be8c-648216b13ebe","Type":"ContainerStarted","Data":"dc5316050aea8ddff8b8d2b9fc17f1896e3a1a6d37edbc5bbb14fd8637106fdb"} Sep 08 18:30:34.163270 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:34.163236 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:34.199409 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:34.199380 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6b68db98f8-5t66l" Sep 08 18:30:37.280636 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:37.280601 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6d985cc74d-b2kg9" event={"ID":"7c1b6bd1-08a0-4fce-be8c-648216b13ebe","Type":"ContainerStarted","Data":"512f5a783ae96a5a05d7103095a4a2c587fb28c9cd533a8e5cd8552cda2aa522"} Sep 08 18:30:37.298278 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:37.298233 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6d985cc74d-b2kg9" podStartSLOduration=3.034112244 podStartE2EDuration="6.298216978s" totalImagesPullingTime="3.264104734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:30:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:30:37.296272393 +0000 UTC m=+419.502257789" watchObservedRunningTime="2026-09-08 18:30:37.298216978 +0000 UTC m=+419.504202374" Sep 08 18:30:39.380560 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:39.380524 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:39.472063 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:39.472035 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:40.234845 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:40.234817 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:40.518737 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:40.518623 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:40.828765 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:40.828725 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:41.960846 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:41.960814 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:42.061279 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:42.061248 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:42.476349 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:42.476310 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:42.568488 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:42.568457 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:43.228343 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:43.228310 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:43.334844 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:43.334810 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:43.481192 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:43.481113 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:30:53.293784 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:53.293707 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-85cd9df9bd-pgq8c" podUID="4fd783b8-b4a1-4f7d-a308-49e85f4be430" containerName="console" containerID="cri-o://73fbd2561a4da214089f3ddccb552abb7192021a73e3e2f8e68e00b638b30b1e" gracePeriod=15 Sep 08 18:30:53.540628 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:53.540604 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85cd9df9bd-pgq8c" Sep 08 18:30:53.671586 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:53.671493 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4fd783b8-b4a1-4f7d-a308-49e85f4be430-oauth-serving-cert\") pod \"4fd783b8-b4a1-4f7d-a308-49e85f4be430\" (UID: \"4fd783b8-b4a1-4f7d-a308-49e85f4be430\") " Sep 08 18:30:53.671586 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:53.671538 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4fd783b8-b4a1-4f7d-a308-49e85f4be430-service-ca\") pod \"4fd783b8-b4a1-4f7d-a308-49e85f4be430\" (UID: \"4fd783b8-b4a1-4f7d-a308-49e85f4be430\") " Sep 08 18:30:53.671586 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:53.671579 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fd783b8-b4a1-4f7d-a308-49e85f4be430-trusted-ca-bundle\") pod \"4fd783b8-b4a1-4f7d-a308-49e85f4be430\" (UID: \"4fd783b8-b4a1-4f7d-a308-49e85f4be430\") " Sep 08 18:30:53.671893 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:53.671608 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4fd783b8-b4a1-4f7d-a308-49e85f4be430-console-serving-cert\") pod \"4fd783b8-b4a1-4f7d-a308-49e85f4be430\" (UID: \"4fd783b8-b4a1-4f7d-a308-49e85f4be430\") " Sep 08 18:30:53.671893 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:53.671670 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-whqkx\" (UniqueName: \"kubernetes.io/projected/4fd783b8-b4a1-4f7d-a308-49e85f4be430-kube-api-access-whqkx\") pod \"4fd783b8-b4a1-4f7d-a308-49e85f4be430\" (UID: \"4fd783b8-b4a1-4f7d-a308-49e85f4be430\") " Sep 08 18:30:53.671893 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:53.671720 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4fd783b8-b4a1-4f7d-a308-49e85f4be430-console-config\") pod \"4fd783b8-b4a1-4f7d-a308-49e85f4be430\" (UID: \"4fd783b8-b4a1-4f7d-a308-49e85f4be430\") " Sep 08 18:30:53.671893 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:53.671777 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4fd783b8-b4a1-4f7d-a308-49e85f4be430-console-oauth-config\") pod \"4fd783b8-b4a1-4f7d-a308-49e85f4be430\" (UID: \"4fd783b8-b4a1-4f7d-a308-49e85f4be430\") " Sep 08 18:30:53.672030 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:53.672013 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fd783b8-b4a1-4f7d-a308-49e85f4be430-service-ca" (OuterVolumeSpecName: "service-ca") pod "4fd783b8-b4a1-4f7d-a308-49e85f4be430" (UID: "4fd783b8-b4a1-4f7d-a308-49e85f4be430"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:30:53.672085 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:53.672050 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fd783b8-b4a1-4f7d-a308-49e85f4be430-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4fd783b8-b4a1-4f7d-a308-49e85f4be430" (UID: "4fd783b8-b4a1-4f7d-a308-49e85f4be430"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:30:53.672119 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:53.672081 2555 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4fd783b8-b4a1-4f7d-a308-49e85f4be430-service-ca\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:30:53.672119 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:53.672094 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fd783b8-b4a1-4f7d-a308-49e85f4be430-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4fd783b8-b4a1-4f7d-a308-49e85f4be430" (UID: "4fd783b8-b4a1-4f7d-a308-49e85f4be430"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:30:53.672180 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:53.672140 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fd783b8-b4a1-4f7d-a308-49e85f4be430-console-config" (OuterVolumeSpecName: "console-config") pod "4fd783b8-b4a1-4f7d-a308-49e85f4be430" (UID: "4fd783b8-b4a1-4f7d-a308-49e85f4be430"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:30:53.674054 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:53.674025 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4fd783b8-b4a1-4f7d-a308-49e85f4be430-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4fd783b8-b4a1-4f7d-a308-49e85f4be430" (UID: "4fd783b8-b4a1-4f7d-a308-49e85f4be430"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 18:30:53.674054 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:53.674041 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fd783b8-b4a1-4f7d-a308-49e85f4be430-kube-api-access-whqkx" (OuterVolumeSpecName: "kube-api-access-whqkx") pod "4fd783b8-b4a1-4f7d-a308-49e85f4be430" (UID: "4fd783b8-b4a1-4f7d-a308-49e85f4be430"). InnerVolumeSpecName "kube-api-access-whqkx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:30:53.674203 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:53.674106 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4fd783b8-b4a1-4f7d-a308-49e85f4be430-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4fd783b8-b4a1-4f7d-a308-49e85f4be430" (UID: "4fd783b8-b4a1-4f7d-a308-49e85f4be430"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 08 18:30:53.772556 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:53.772518 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whqkx\" (UniqueName: \"kubernetes.io/projected/4fd783b8-b4a1-4f7d-a308-49e85f4be430-kube-api-access-whqkx\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:30:53.772556 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:53.772551 2555 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4fd783b8-b4a1-4f7d-a308-49e85f4be430-console-config\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:30:53.772556 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:53.772560 2555 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4fd783b8-b4a1-4f7d-a308-49e85f4be430-console-oauth-config\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:30:53.772821 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:53.772568 2555 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4fd783b8-b4a1-4f7d-a308-49e85f4be430-oauth-serving-cert\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:30:53.772821 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:53.772580 2555 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fd783b8-b4a1-4f7d-a308-49e85f4be430-trusted-ca-bundle\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:30:53.772821 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:53.772588 2555 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4fd783b8-b4a1-4f7d-a308-49e85f4be430-console-serving-cert\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:30:54.365278 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:54.365247 2555 generic.go:356] "Generic (PLEG): container finished" podID="4fd783b8-b4a1-4f7d-a308-49e85f4be430" containerID="73fbd2561a4da214089f3ddccb552abb7192021a73e3e2f8e68e00b638b30b1e" exitCode=0 Sep 08 18:30:54.365670 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:54.365300 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85cd9df9bd-pgq8c" Sep 08 18:30:54.365670 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:54.365330 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85cd9df9bd-pgq8c" event={"ID":"4fd783b8-b4a1-4f7d-a308-49e85f4be430","Type":"ContainerDied","Data":"73fbd2561a4da214089f3ddccb552abb7192021a73e3e2f8e68e00b638b30b1e"} Sep 08 18:30:54.365670 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:54.365385 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85cd9df9bd-pgq8c" event={"ID":"4fd783b8-b4a1-4f7d-a308-49e85f4be430","Type":"ContainerDied","Data":"682e44f239c32ea4c84400d566df935abc0565d54250092fa78fa41d509d6c09"} Sep 08 18:30:54.365670 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:54.365414 2555 scope.go:117] "RemoveContainer" containerID="73fbd2561a4da214089f3ddccb552abb7192021a73e3e2f8e68e00b638b30b1e" Sep 08 18:30:54.374013 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:54.373987 2555 scope.go:117] "RemoveContainer" containerID="73fbd2561a4da214089f3ddccb552abb7192021a73e3e2f8e68e00b638b30b1e" Sep 08 18:30:54.374264 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:30:54.374244 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73fbd2561a4da214089f3ddccb552abb7192021a73e3e2f8e68e00b638b30b1e\": container with ID starting with 73fbd2561a4da214089f3ddccb552abb7192021a73e3e2f8e68e00b638b30b1e not found: ID does not exist" containerID="73fbd2561a4da214089f3ddccb552abb7192021a73e3e2f8e68e00b638b30b1e" Sep 08 18:30:54.374327 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:54.374275 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73fbd2561a4da214089f3ddccb552abb7192021a73e3e2f8e68e00b638b30b1e"} err="failed to get container status \"73fbd2561a4da214089f3ddccb552abb7192021a73e3e2f8e68e00b638b30b1e\": rpc error: code = NotFound desc = could not find container \"73fbd2561a4da214089f3ddccb552abb7192021a73e3e2f8e68e00b638b30b1e\": container with ID starting with 73fbd2561a4da214089f3ddccb552abb7192021a73e3e2f8e68e00b638b30b1e not found: ID does not exist" Sep 08 18:30:54.380787 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:54.380765 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85cd9df9bd-pgq8c"] Sep 08 18:30:54.385169 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:54.385149 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-85cd9df9bd-pgq8c"] Sep 08 18:30:56.323596 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:56.323562 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4fd783b8-b4a1-4f7d-a308-49e85f4be430" path="/var/lib/kubelet/pods/4fd783b8-b4a1-4f7d-a308-49e85f4be430/volumes" Sep 08 18:30:56.977564 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:30:56.977496 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:31:32.999013 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:32.998974 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:31:33.102347 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:33.102309 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:31:36.722008 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:36.721972 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lc7k2"] Sep 08 18:31:36.722404 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:36.722336 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fd783b8-b4a1-4f7d-a308-49e85f4be430" containerName="console" Sep 08 18:31:36.722404 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:36.722347 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fd783b8-b4a1-4f7d-a308-49e85f4be430" containerName="console" Sep 08 18:31:36.722404 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:36.722403 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fd783b8-b4a1-4f7d-a308-49e85f4be430" containerName="console" Sep 08 18:31:36.725657 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:36.725640 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lc7k2" Sep 08 18:31:36.727952 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:36.727929 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 08 18:31:36.728353 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:36.728337 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-4xn48\"" Sep 08 18:31:36.728440 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:36.728342 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 08 18:31:36.733251 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:36.732917 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lc7k2"] Sep 08 18:31:36.815914 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:36.815877 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hb5m"] Sep 08 18:31:36.819523 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:36.819501 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hb5m" Sep 08 18:31:36.827534 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:36.827511 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hb5m"] Sep 08 18:31:36.845009 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:36.844979 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjq8l\" (UniqueName: \"kubernetes.io/projected/02e7edc5-9a02-4eb1-b05e-f7f10f461da6-kube-api-access-kjq8l\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lc7k2\" (UID: \"02e7edc5-9a02-4eb1-b05e-f7f10f461da6\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lc7k2" Sep 08 18:31:36.845121 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:36.845016 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/02e7edc5-9a02-4eb1-b05e-f7f10f461da6-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lc7k2\" (UID: \"02e7edc5-9a02-4eb1-b05e-f7f10f461da6\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lc7k2" Sep 08 18:31:36.845121 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:36.845050 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/02e7edc5-9a02-4eb1-b05e-f7f10f461da6-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lc7k2\" (UID: \"02e7edc5-9a02-4eb1-b05e-f7f10f461da6\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lc7k2" Sep 08 18:31:36.924129 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:36.924098 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctg6qs"] Sep 08 18:31:36.927752 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:36.927738 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctg6qs" Sep 08 18:31:36.936736 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:36.936709 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctg6qs"] Sep 08 18:31:36.946150 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:36.946126 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/85ddbfff-5b0c-4d54-97d0-0915bed12562-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hb5m\" (UID: \"85ddbfff-5b0c-4d54-97d0-0915bed12562\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hb5m" Sep 08 18:31:36.946267 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:36.946173 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgq7l\" (UniqueName: \"kubernetes.io/projected/85ddbfff-5b0c-4d54-97d0-0915bed12562-kube-api-access-xgq7l\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hb5m\" (UID: \"85ddbfff-5b0c-4d54-97d0-0915bed12562\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hb5m" Sep 08 18:31:36.946267 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:36.946220 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/85ddbfff-5b0c-4d54-97d0-0915bed12562-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hb5m\" (UID: \"85ddbfff-5b0c-4d54-97d0-0915bed12562\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hb5m" Sep 08 18:31:36.946267 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:36.946251 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjq8l\" (UniqueName: \"kubernetes.io/projected/02e7edc5-9a02-4eb1-b05e-f7f10f461da6-kube-api-access-kjq8l\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lc7k2\" (UID: \"02e7edc5-9a02-4eb1-b05e-f7f10f461da6\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lc7k2" Sep 08 18:31:36.946430 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:36.946270 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/02e7edc5-9a02-4eb1-b05e-f7f10f461da6-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lc7k2\" (UID: \"02e7edc5-9a02-4eb1-b05e-f7f10f461da6\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lc7k2" Sep 08 18:31:36.946430 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:36.946355 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/02e7edc5-9a02-4eb1-b05e-f7f10f461da6-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lc7k2\" (UID: \"02e7edc5-9a02-4eb1-b05e-f7f10f461da6\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lc7k2" Sep 08 18:31:36.946632 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:36.946612 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/02e7edc5-9a02-4eb1-b05e-f7f10f461da6-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lc7k2\" (UID: \"02e7edc5-9a02-4eb1-b05e-f7f10f461da6\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lc7k2" Sep 08 18:31:36.946632 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:36.946625 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/02e7edc5-9a02-4eb1-b05e-f7f10f461da6-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lc7k2\" (UID: \"02e7edc5-9a02-4eb1-b05e-f7f10f461da6\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lc7k2" Sep 08 18:31:36.955777 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:36.955746 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjq8l\" (UniqueName: \"kubernetes.io/projected/02e7edc5-9a02-4eb1-b05e-f7f10f461da6-kube-api-access-kjq8l\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lc7k2\" (UID: \"02e7edc5-9a02-4eb1-b05e-f7f10f461da6\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lc7k2" Sep 08 18:31:37.025630 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.025524 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79msd7"] Sep 08 18:31:37.029434 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.029419 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79msd7" Sep 08 18:31:37.037310 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.037287 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lc7k2" Sep 08 18:31:37.037844 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.037825 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79msd7"] Sep 08 18:31:37.048153 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.048133 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfzpj\" (UniqueName: \"kubernetes.io/projected/c74608dd-db3f-42d1-bd92-9855edc26559-kube-api-access-zfzpj\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctg6qs\" (UID: \"c74608dd-db3f-42d1-bd92-9855edc26559\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctg6qs" Sep 08 18:31:37.048253 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.048181 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c74608dd-db3f-42d1-bd92-9855edc26559-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctg6qs\" (UID: \"c74608dd-db3f-42d1-bd92-9855edc26559\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctg6qs" Sep 08 18:31:37.048253 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.048217 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/85ddbfff-5b0c-4d54-97d0-0915bed12562-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hb5m\" (UID: \"85ddbfff-5b0c-4d54-97d0-0915bed12562\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hb5m" Sep 08 18:31:37.048368 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.048317 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c74608dd-db3f-42d1-bd92-9855edc26559-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctg6qs\" (UID: \"c74608dd-db3f-42d1-bd92-9855edc26559\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctg6qs" Sep 08 18:31:37.048422 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.048361 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgq7l\" (UniqueName: \"kubernetes.io/projected/85ddbfff-5b0c-4d54-97d0-0915bed12562-kube-api-access-xgq7l\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hb5m\" (UID: \"85ddbfff-5b0c-4d54-97d0-0915bed12562\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hb5m" Sep 08 18:31:37.048422 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.048411 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/85ddbfff-5b0c-4d54-97d0-0915bed12562-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hb5m\" (UID: \"85ddbfff-5b0c-4d54-97d0-0915bed12562\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hb5m" Sep 08 18:31:37.048638 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.048524 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/85ddbfff-5b0c-4d54-97d0-0915bed12562-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hb5m\" (UID: \"85ddbfff-5b0c-4d54-97d0-0915bed12562\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hb5m" Sep 08 18:31:37.048750 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.048733 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/85ddbfff-5b0c-4d54-97d0-0915bed12562-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hb5m\" (UID: \"85ddbfff-5b0c-4d54-97d0-0915bed12562\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hb5m" Sep 08 18:31:37.056550 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.056521 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgq7l\" (UniqueName: \"kubernetes.io/projected/85ddbfff-5b0c-4d54-97d0-0915bed12562-kube-api-access-xgq7l\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hb5m\" (UID: \"85ddbfff-5b0c-4d54-97d0-0915bed12562\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hb5m" Sep 08 18:31:37.129360 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.129330 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hb5m" Sep 08 18:31:37.149780 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.149752 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfzpj\" (UniqueName: \"kubernetes.io/projected/c74608dd-db3f-42d1-bd92-9855edc26559-kube-api-access-zfzpj\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctg6qs\" (UID: \"c74608dd-db3f-42d1-bd92-9855edc26559\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctg6qs" Sep 08 18:31:37.149910 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.149791 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9ebe3c3-e921-4f7e-8724-48bbcdd02e40-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79msd7\" (UID: \"e9ebe3c3-e921-4f7e-8724-48bbcdd02e40\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79msd7" Sep 08 18:31:37.149910 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.149840 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c74608dd-db3f-42d1-bd92-9855edc26559-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctg6qs\" (UID: \"c74608dd-db3f-42d1-bd92-9855edc26559\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctg6qs" Sep 08 18:31:37.149910 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.149859 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9ebe3c3-e921-4f7e-8724-48bbcdd02e40-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79msd7\" (UID: \"e9ebe3c3-e921-4f7e-8724-48bbcdd02e40\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79msd7" Sep 08 18:31:37.149910 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.149883 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cswvx\" (UniqueName: \"kubernetes.io/projected/e9ebe3c3-e921-4f7e-8724-48bbcdd02e40-kube-api-access-cswvx\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79msd7\" (UID: \"e9ebe3c3-e921-4f7e-8724-48bbcdd02e40\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79msd7" Sep 08 18:31:37.150052 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.149927 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c74608dd-db3f-42d1-bd92-9855edc26559-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctg6qs\" (UID: \"c74608dd-db3f-42d1-bd92-9855edc26559\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctg6qs" Sep 08 18:31:37.150284 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.150262 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c74608dd-db3f-42d1-bd92-9855edc26559-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctg6qs\" (UID: \"c74608dd-db3f-42d1-bd92-9855edc26559\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctg6qs" Sep 08 18:31:37.150374 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.150268 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c74608dd-db3f-42d1-bd92-9855edc26559-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctg6qs\" (UID: \"c74608dd-db3f-42d1-bd92-9855edc26559\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctg6qs" Sep 08 18:31:37.157808 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.157785 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfzpj\" (UniqueName: \"kubernetes.io/projected/c74608dd-db3f-42d1-bd92-9855edc26559-kube-api-access-zfzpj\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctg6qs\" (UID: \"c74608dd-db3f-42d1-bd92-9855edc26559\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctg6qs" Sep 08 18:31:37.237067 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.237033 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctg6qs" Sep 08 18:31:37.251265 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.251241 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9ebe3c3-e921-4f7e-8724-48bbcdd02e40-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79msd7\" (UID: \"e9ebe3c3-e921-4f7e-8724-48bbcdd02e40\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79msd7" Sep 08 18:31:37.251339 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.251286 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cswvx\" (UniqueName: \"kubernetes.io/projected/e9ebe3c3-e921-4f7e-8724-48bbcdd02e40-kube-api-access-cswvx\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79msd7\" (UID: \"e9ebe3c3-e921-4f7e-8724-48bbcdd02e40\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79msd7" Sep 08 18:31:37.251396 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.251373 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9ebe3c3-e921-4f7e-8724-48bbcdd02e40-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79msd7\" (UID: \"e9ebe3c3-e921-4f7e-8724-48bbcdd02e40\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79msd7" Sep 08 18:31:37.251606 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.251580 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9ebe3c3-e921-4f7e-8724-48bbcdd02e40-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79msd7\" (UID: \"e9ebe3c3-e921-4f7e-8724-48bbcdd02e40\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79msd7" Sep 08 18:31:37.251736 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.251631 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9ebe3c3-e921-4f7e-8724-48bbcdd02e40-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79msd7\" (UID: \"e9ebe3c3-e921-4f7e-8724-48bbcdd02e40\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79msd7" Sep 08 18:31:37.260395 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.260374 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cswvx\" (UniqueName: \"kubernetes.io/projected/e9ebe3c3-e921-4f7e-8724-48bbcdd02e40-kube-api-access-cswvx\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79msd7\" (UID: \"e9ebe3c3-e921-4f7e-8724-48bbcdd02e40\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79msd7" Sep 08 18:31:37.313025 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.312995 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lc7k2"] Sep 08 18:31:37.315809 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:31:37.315781 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02e7edc5_9a02_4eb1_b05e_f7f10f461da6.slice/crio-ab405ba06434e791ddef09b48f763022db42af633e19934d1bb64f08cc8f5e4a WatchSource:0}: Error finding container ab405ba06434e791ddef09b48f763022db42af633e19934d1bb64f08cc8f5e4a: Status 404 returned error can't find the container with id ab405ba06434e791ddef09b48f763022db42af633e19934d1bb64f08cc8f5e4a Sep 08 18:31:37.339078 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.339048 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79msd7" Sep 08 18:31:37.402451 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.402405 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hb5m"] Sep 08 18:31:37.407879 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:31:37.407853 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85ddbfff_5b0c_4d54_97d0_0915bed12562.slice/crio-7eb96ab4ab2e0fcc7dee9686eb134a531a5ef2449a55eb2501112495407b4d4f WatchSource:0}: Error finding container 7eb96ab4ab2e0fcc7dee9686eb134a531a5ef2449a55eb2501112495407b4d4f: Status 404 returned error can't find the container with id 7eb96ab4ab2e0fcc7dee9686eb134a531a5ef2449a55eb2501112495407b4d4f Sep 08 18:31:37.509132 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.509105 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctg6qs"] Sep 08 18:31:37.510417 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:31:37.510391 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc74608dd_db3f_42d1_bd92_9855edc26559.slice/crio-613f248b0e332c570dc726bf47fcf66e5da6f9b2285f82681fbb3530fff18a99 WatchSource:0}: Error finding container 613f248b0e332c570dc726bf47fcf66e5da6f9b2285f82681fbb3530fff18a99: Status 404 returned error can't find the container with id 613f248b0e332c570dc726bf47fcf66e5da6f9b2285f82681fbb3530fff18a99 Sep 08 18:31:37.579363 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.579327 2555 generic.go:356] "Generic (PLEG): container finished" podID="85ddbfff-5b0c-4d54-97d0-0915bed12562" containerID="ed2c6facf083c812dc7daea69add9926b647a8f1c637a08015ead84453bfdafa" exitCode=0 Sep 08 18:31:37.579552 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.579419 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hb5m" event={"ID":"85ddbfff-5b0c-4d54-97d0-0915bed12562","Type":"ContainerDied","Data":"ed2c6facf083c812dc7daea69add9926b647a8f1c637a08015ead84453bfdafa"} Sep 08 18:31:37.579552 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.579467 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hb5m" event={"ID":"85ddbfff-5b0c-4d54-97d0-0915bed12562","Type":"ContainerStarted","Data":"7eb96ab4ab2e0fcc7dee9686eb134a531a5ef2449a55eb2501112495407b4d4f"} Sep 08 18:31:37.580792 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.580760 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctg6qs" event={"ID":"c74608dd-db3f-42d1-bd92-9855edc26559","Type":"ContainerStarted","Data":"613f248b0e332c570dc726bf47fcf66e5da6f9b2285f82681fbb3530fff18a99"} Sep 08 18:31:37.582432 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.582406 2555 generic.go:356] "Generic (PLEG): container finished" podID="02e7edc5-9a02-4eb1-b05e-f7f10f461da6" containerID="15c6f2d2496ae645ecb9ea622a4f8fce4797c99e8f9f8c9fb812a3fb263854f2" exitCode=0 Sep 08 18:31:37.582545 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.582499 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lc7k2" event={"ID":"02e7edc5-9a02-4eb1-b05e-f7f10f461da6","Type":"ContainerDied","Data":"15c6f2d2496ae645ecb9ea622a4f8fce4797c99e8f9f8c9fb812a3fb263854f2"} Sep 08 18:31:37.582612 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.582541 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lc7k2" event={"ID":"02e7edc5-9a02-4eb1-b05e-f7f10f461da6","Type":"ContainerStarted","Data":"ab405ba06434e791ddef09b48f763022db42af633e19934d1bb64f08cc8f5e4a"} Sep 08 18:31:37.617848 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:37.617822 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79msd7"] Sep 08 18:31:37.620299 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:31:37.620276 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9ebe3c3_e921_4f7e_8724_48bbcdd02e40.slice/crio-56a8ee692c38c9a692a6836178a0d0f460dfb1c5a97cd10d1c2adf2b702d7fbe WatchSource:0}: Error finding container 56a8ee692c38c9a692a6836178a0d0f460dfb1c5a97cd10d1c2adf2b702d7fbe: Status 404 returned error can't find the container with id 56a8ee692c38c9a692a6836178a0d0f460dfb1c5a97cd10d1c2adf2b702d7fbe Sep 08 18:31:38.590145 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:38.590115 2555 generic.go:356] "Generic (PLEG): container finished" podID="c74608dd-db3f-42d1-bd92-9855edc26559" containerID="c54ffac6b27675f976291d3dab70adbe9e7c68245fd90986c373c47558f71b35" exitCode=0 Sep 08 18:31:38.590588 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:38.590228 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctg6qs" event={"ID":"c74608dd-db3f-42d1-bd92-9855edc26559","Type":"ContainerDied","Data":"c54ffac6b27675f976291d3dab70adbe9e7c68245fd90986c373c47558f71b35"} Sep 08 18:31:38.592402 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:38.592376 2555 generic.go:356] "Generic (PLEG): container finished" podID="02e7edc5-9a02-4eb1-b05e-f7f10f461da6" containerID="3e769eb3dd0264ccc7446ca26fa3fb6f3121e9b7a5499e47a4e18384aec14fd5" exitCode=0 Sep 08 18:31:38.592726 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:38.592448 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lc7k2" event={"ID":"02e7edc5-9a02-4eb1-b05e-f7f10f461da6","Type":"ContainerDied","Data":"3e769eb3dd0264ccc7446ca26fa3fb6f3121e9b7a5499e47a4e18384aec14fd5"} Sep 08 18:31:38.594421 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:38.594399 2555 generic.go:356] "Generic (PLEG): container finished" podID="e9ebe3c3-e921-4f7e-8724-48bbcdd02e40" containerID="453995f5571549a5e62eff30e0d96496b6ddd2e9f680451d964d2a3fb5576609" exitCode=0 Sep 08 18:31:38.594516 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:38.594475 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79msd7" event={"ID":"e9ebe3c3-e921-4f7e-8724-48bbcdd02e40","Type":"ContainerDied","Data":"453995f5571549a5e62eff30e0d96496b6ddd2e9f680451d964d2a3fb5576609"} Sep 08 18:31:38.594586 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:38.594522 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79msd7" event={"ID":"e9ebe3c3-e921-4f7e-8724-48bbcdd02e40","Type":"ContainerStarted","Data":"56a8ee692c38c9a692a6836178a0d0f460dfb1c5a97cd10d1c2adf2b702d7fbe"} Sep 08 18:31:39.605954 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:39.605928 2555 generic.go:356] "Generic (PLEG): container finished" podID="02e7edc5-9a02-4eb1-b05e-f7f10f461da6" containerID="055f84281022a4d4bf19366b386963362a91a922585695130460c3941e2dbb58" exitCode=0 Sep 08 18:31:39.606265 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:39.606021 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lc7k2" event={"ID":"02e7edc5-9a02-4eb1-b05e-f7f10f461da6","Type":"ContainerDied","Data":"055f84281022a4d4bf19366b386963362a91a922585695130460c3941e2dbb58"} Sep 08 18:31:39.607727 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:39.607704 2555 generic.go:356] "Generic (PLEG): container finished" podID="e9ebe3c3-e921-4f7e-8724-48bbcdd02e40" containerID="45f14a70a501f0ef0edf538bc1c6c0c09e7b59ff0fc1d5ec6631fdbbc9ba5f2e" exitCode=0 Sep 08 18:31:39.607802 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:39.607783 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79msd7" event={"ID":"e9ebe3c3-e921-4f7e-8724-48bbcdd02e40","Type":"ContainerDied","Data":"45f14a70a501f0ef0edf538bc1c6c0c09e7b59ff0fc1d5ec6631fdbbc9ba5f2e"} Sep 08 18:31:39.609733 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:39.609711 2555 generic.go:356] "Generic (PLEG): container finished" podID="85ddbfff-5b0c-4d54-97d0-0915bed12562" containerID="2662c9544fe9c6af19cca1031b8197a71f442091bc7a084166ba1bca0db5fb47" exitCode=0 Sep 08 18:31:39.609869 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:39.609790 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hb5m" event={"ID":"85ddbfff-5b0c-4d54-97d0-0915bed12562","Type":"ContainerDied","Data":"2662c9544fe9c6af19cca1031b8197a71f442091bc7a084166ba1bca0db5fb47"} Sep 08 18:31:39.611709 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:39.611674 2555 generic.go:356] "Generic (PLEG): container finished" podID="c74608dd-db3f-42d1-bd92-9855edc26559" containerID="a018b94e4258d877e1df31e2ef58a83ba4e29f114af8386bfeca095cd1aef65b" exitCode=0 Sep 08 18:31:39.611795 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:39.611725 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctg6qs" event={"ID":"c74608dd-db3f-42d1-bd92-9855edc26559","Type":"ContainerDied","Data":"a018b94e4258d877e1df31e2ef58a83ba4e29f114af8386bfeca095cd1aef65b"} Sep 08 18:31:40.618495 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:40.618456 2555 generic.go:356] "Generic (PLEG): container finished" podID="85ddbfff-5b0c-4d54-97d0-0915bed12562" containerID="07674b8cf764c729fa90e938d72d0b31825d172771b58d61e6b450e2f5fd6aad" exitCode=0 Sep 08 18:31:40.618995 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:40.618535 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hb5m" event={"ID":"85ddbfff-5b0c-4d54-97d0-0915bed12562","Type":"ContainerDied","Data":"07674b8cf764c729fa90e938d72d0b31825d172771b58d61e6b450e2f5fd6aad"} Sep 08 18:31:40.620588 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:40.620569 2555 generic.go:356] "Generic (PLEG): container finished" podID="c74608dd-db3f-42d1-bd92-9855edc26559" containerID="8eb5a48e91b667f0940a9afeebca613763a2582f2d306d210722f4db08cf152c" exitCode=0 Sep 08 18:31:40.620733 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:40.620649 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctg6qs" event={"ID":"c74608dd-db3f-42d1-bd92-9855edc26559","Type":"ContainerDied","Data":"8eb5a48e91b667f0940a9afeebca613763a2582f2d306d210722f4db08cf152c"} Sep 08 18:31:40.622895 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:40.622877 2555 generic.go:356] "Generic (PLEG): container finished" podID="e9ebe3c3-e921-4f7e-8724-48bbcdd02e40" containerID="7adb4819e4da98289b940e1158d20b7ccbc7ddfe32b661931ef27f4bc85839c4" exitCode=0 Sep 08 18:31:40.622988 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:40.622904 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79msd7" event={"ID":"e9ebe3c3-e921-4f7e-8724-48bbcdd02e40","Type":"ContainerDied","Data":"7adb4819e4da98289b940e1158d20b7ccbc7ddfe32b661931ef27f4bc85839c4"} Sep 08 18:31:40.759587 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:40.759566 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lc7k2" Sep 08 18:31:40.883680 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:40.883597 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjq8l\" (UniqueName: \"kubernetes.io/projected/02e7edc5-9a02-4eb1-b05e-f7f10f461da6-kube-api-access-kjq8l\") pod \"02e7edc5-9a02-4eb1-b05e-f7f10f461da6\" (UID: \"02e7edc5-9a02-4eb1-b05e-f7f10f461da6\") " Sep 08 18:31:40.883680 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:40.883670 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/02e7edc5-9a02-4eb1-b05e-f7f10f461da6-bundle\") pod \"02e7edc5-9a02-4eb1-b05e-f7f10f461da6\" (UID: \"02e7edc5-9a02-4eb1-b05e-f7f10f461da6\") " Sep 08 18:31:40.883910 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:40.883714 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/02e7edc5-9a02-4eb1-b05e-f7f10f461da6-util\") pod \"02e7edc5-9a02-4eb1-b05e-f7f10f461da6\" (UID: \"02e7edc5-9a02-4eb1-b05e-f7f10f461da6\") " Sep 08 18:31:40.884066 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:40.884031 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02e7edc5-9a02-4eb1-b05e-f7f10f461da6-bundle" (OuterVolumeSpecName: "bundle") pod "02e7edc5-9a02-4eb1-b05e-f7f10f461da6" (UID: "02e7edc5-9a02-4eb1-b05e-f7f10f461da6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:31:40.885896 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:40.885872 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02e7edc5-9a02-4eb1-b05e-f7f10f461da6-kube-api-access-kjq8l" (OuterVolumeSpecName: "kube-api-access-kjq8l") pod "02e7edc5-9a02-4eb1-b05e-f7f10f461da6" (UID: "02e7edc5-9a02-4eb1-b05e-f7f10f461da6"). InnerVolumeSpecName "kube-api-access-kjq8l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:31:40.889235 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:40.889212 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02e7edc5-9a02-4eb1-b05e-f7f10f461da6-util" (OuterVolumeSpecName: "util") pod "02e7edc5-9a02-4eb1-b05e-f7f10f461da6" (UID: "02e7edc5-9a02-4eb1-b05e-f7f10f461da6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:31:40.984494 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:40.984458 2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/02e7edc5-9a02-4eb1-b05e-f7f10f461da6-bundle\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:31:40.984494 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:40.984486 2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/02e7edc5-9a02-4eb1-b05e-f7f10f461da6-util\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:31:40.984494 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:40.984497 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjq8l\" (UniqueName: \"kubernetes.io/projected/02e7edc5-9a02-4eb1-b05e-f7f10f461da6-kube-api-access-kjq8l\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:31:41.629567 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:41.629527 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lc7k2" Sep 08 18:31:41.629567 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:41.629552 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6lc7k2" event={"ID":"02e7edc5-9a02-4eb1-b05e-f7f10f461da6","Type":"ContainerDied","Data":"ab405ba06434e791ddef09b48f763022db42af633e19934d1bb64f08cc8f5e4a"} Sep 08 18:31:41.630026 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:41.629622 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ab405ba06434e791ddef09b48f763022db42af633e19934d1bb64f08cc8f5e4a" Sep 08 18:31:41.795081 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:41.795056 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctg6qs" Sep 08 18:31:41.798393 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:41.798370 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79msd7" Sep 08 18:31:41.807737 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:41.807717 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hb5m" Sep 08 18:31:41.893061 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:41.892977 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgq7l\" (UniqueName: \"kubernetes.io/projected/85ddbfff-5b0c-4d54-97d0-0915bed12562-kube-api-access-xgq7l\") pod \"85ddbfff-5b0c-4d54-97d0-0915bed12562\" (UID: \"85ddbfff-5b0c-4d54-97d0-0915bed12562\") " Sep 08 18:31:41.893061 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:41.893032 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c74608dd-db3f-42d1-bd92-9855edc26559-util\") pod \"c74608dd-db3f-42d1-bd92-9855edc26559\" (UID: \"c74608dd-db3f-42d1-bd92-9855edc26559\") " Sep 08 18:31:41.893270 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:41.893064 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9ebe3c3-e921-4f7e-8724-48bbcdd02e40-bundle\") pod \"e9ebe3c3-e921-4f7e-8724-48bbcdd02e40\" (UID: \"e9ebe3c3-e921-4f7e-8724-48bbcdd02e40\") " Sep 08 18:31:41.893270 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:41.893187 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/85ddbfff-5b0c-4d54-97d0-0915bed12562-bundle\") pod \"85ddbfff-5b0c-4d54-97d0-0915bed12562\" (UID: \"85ddbfff-5b0c-4d54-97d0-0915bed12562\") " Sep 08 18:31:41.893270 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:41.893237 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cswvx\" (UniqueName: \"kubernetes.io/projected/e9ebe3c3-e921-4f7e-8724-48bbcdd02e40-kube-api-access-cswvx\") pod \"e9ebe3c3-e921-4f7e-8724-48bbcdd02e40\" (UID: \"e9ebe3c3-e921-4f7e-8724-48bbcdd02e40\") " Sep 08 18:31:41.893425 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:41.893271 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zfzpj\" (UniqueName: \"kubernetes.io/projected/c74608dd-db3f-42d1-bd92-9855edc26559-kube-api-access-zfzpj\") pod \"c74608dd-db3f-42d1-bd92-9855edc26559\" (UID: \"c74608dd-db3f-42d1-bd92-9855edc26559\") " Sep 08 18:31:41.893425 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:41.893318 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c74608dd-db3f-42d1-bd92-9855edc26559-bundle\") pod \"c74608dd-db3f-42d1-bd92-9855edc26559\" (UID: \"c74608dd-db3f-42d1-bd92-9855edc26559\") " Sep 08 18:31:41.893425 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:41.893364 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/85ddbfff-5b0c-4d54-97d0-0915bed12562-util\") pod \"85ddbfff-5b0c-4d54-97d0-0915bed12562\" (UID: \"85ddbfff-5b0c-4d54-97d0-0915bed12562\") " Sep 08 18:31:41.893425 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:41.893403 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9ebe3c3-e921-4f7e-8724-48bbcdd02e40-util\") pod \"e9ebe3c3-e921-4f7e-8724-48bbcdd02e40\" (UID: \"e9ebe3c3-e921-4f7e-8724-48bbcdd02e40\") " Sep 08 18:31:41.894025 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:41.893982 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9ebe3c3-e921-4f7e-8724-48bbcdd02e40-bundle" (OuterVolumeSpecName: "bundle") pod "e9ebe3c3-e921-4f7e-8724-48bbcdd02e40" (UID: "e9ebe3c3-e921-4f7e-8724-48bbcdd02e40"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:31:41.894444 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:41.894367 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c74608dd-db3f-42d1-bd92-9855edc26559-bundle" (OuterVolumeSpecName: "bundle") pod "c74608dd-db3f-42d1-bd92-9855edc26559" (UID: "c74608dd-db3f-42d1-bd92-9855edc26559"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:31:41.894444 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:41.894398 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85ddbfff-5b0c-4d54-97d0-0915bed12562-bundle" (OuterVolumeSpecName: "bundle") pod "85ddbfff-5b0c-4d54-97d0-0915bed12562" (UID: "85ddbfff-5b0c-4d54-97d0-0915bed12562"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:31:41.896143 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:41.896118 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85ddbfff-5b0c-4d54-97d0-0915bed12562-kube-api-access-xgq7l" (OuterVolumeSpecName: "kube-api-access-xgq7l") pod "85ddbfff-5b0c-4d54-97d0-0915bed12562" (UID: "85ddbfff-5b0c-4d54-97d0-0915bed12562"). InnerVolumeSpecName "kube-api-access-xgq7l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:31:41.896211 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:41.896123 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9ebe3c3-e921-4f7e-8724-48bbcdd02e40-kube-api-access-cswvx" (OuterVolumeSpecName: "kube-api-access-cswvx") pod "e9ebe3c3-e921-4f7e-8724-48bbcdd02e40" (UID: "e9ebe3c3-e921-4f7e-8724-48bbcdd02e40"). InnerVolumeSpecName "kube-api-access-cswvx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:31:41.896562 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:41.896544 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c74608dd-db3f-42d1-bd92-9855edc26559-kube-api-access-zfzpj" (OuterVolumeSpecName: "kube-api-access-zfzpj") pod "c74608dd-db3f-42d1-bd92-9855edc26559" (UID: "c74608dd-db3f-42d1-bd92-9855edc26559"). InnerVolumeSpecName "kube-api-access-zfzpj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:31:41.900320 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:41.900301 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85ddbfff-5b0c-4d54-97d0-0915bed12562-util" (OuterVolumeSpecName: "util") pod "85ddbfff-5b0c-4d54-97d0-0915bed12562" (UID: "85ddbfff-5b0c-4d54-97d0-0915bed12562"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:31:41.901739 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:41.901718 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9ebe3c3-e921-4f7e-8724-48bbcdd02e40-util" (OuterVolumeSpecName: "util") pod "e9ebe3c3-e921-4f7e-8724-48bbcdd02e40" (UID: "e9ebe3c3-e921-4f7e-8724-48bbcdd02e40"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:31:41.902090 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:41.902076 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c74608dd-db3f-42d1-bd92-9855edc26559-util" (OuterVolumeSpecName: "util") pod "c74608dd-db3f-42d1-bd92-9855edc26559" (UID: "c74608dd-db3f-42d1-bd92-9855edc26559"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:31:41.994274 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:41.994235 2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c74608dd-db3f-42d1-bd92-9855edc26559-bundle\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:31:41.994274 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:41.994269 2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/85ddbfff-5b0c-4d54-97d0-0915bed12562-util\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:31:41.994274 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:41.994277 2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9ebe3c3-e921-4f7e-8724-48bbcdd02e40-util\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:31:41.994518 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:41.994285 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgq7l\" (UniqueName: \"kubernetes.io/projected/85ddbfff-5b0c-4d54-97d0-0915bed12562-kube-api-access-xgq7l\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:31:41.994518 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:41.994295 2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c74608dd-db3f-42d1-bd92-9855edc26559-util\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:31:41.994518 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:41.994303 2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9ebe3c3-e921-4f7e-8724-48bbcdd02e40-bundle\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:31:41.994518 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:41.994309 2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/85ddbfff-5b0c-4d54-97d0-0915bed12562-bundle\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:31:41.994518 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:41.994317 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cswvx\" (UniqueName: \"kubernetes.io/projected/e9ebe3c3-e921-4f7e-8724-48bbcdd02e40-kube-api-access-cswvx\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:31:41.994518 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:41.994325 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfzpj\" (UniqueName: \"kubernetes.io/projected/c74608dd-db3f-42d1-bd92-9855edc26559-kube-api-access-zfzpj\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:31:42.636840 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:42.636755 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hb5m" Sep 08 18:31:42.636840 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:42.636770 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a6hb5m" event={"ID":"85ddbfff-5b0c-4d54-97d0-0915bed12562","Type":"ContainerDied","Data":"7eb96ab4ab2e0fcc7dee9686eb134a531a5ef2449a55eb2501112495407b4d4f"} Sep 08 18:31:42.636840 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:42.636826 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7eb96ab4ab2e0fcc7dee9686eb134a531a5ef2449a55eb2501112495407b4d4f" Sep 08 18:31:42.638849 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:42.638828 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctg6qs" Sep 08 18:31:42.638849 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:42.638833 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctg6qs" event={"ID":"c74608dd-db3f-42d1-bd92-9855edc26559","Type":"ContainerDied","Data":"613f248b0e332c570dc726bf47fcf66e5da6f9b2285f82681fbb3530fff18a99"} Sep 08 18:31:42.639039 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:42.638874 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="613f248b0e332c570dc726bf47fcf66e5da6f9b2285f82681fbb3530fff18a99" Sep 08 18:31:42.640992 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:42.640969 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79msd7" event={"ID":"e9ebe3c3-e921-4f7e-8724-48bbcdd02e40","Type":"ContainerDied","Data":"56a8ee692c38c9a692a6836178a0d0f460dfb1c5a97cd10d1c2adf2b702d7fbe"} Sep 08 18:31:42.640992 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:42.640989 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79msd7" Sep 08 18:31:42.641156 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:42.641011 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="56a8ee692c38c9a692a6836178a0d0f460dfb1c5a97cd10d1c2adf2b702d7fbe" Sep 08 18:31:44.092365 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:44.092327 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:31:44.214988 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:44.214948 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:31:44.315529 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:44.315493 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:31:44.890053 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:44.890020 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:31:45.690034 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:45.690000 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:31:48.698821 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:48.698791 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:31:48.966992 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:48.966913 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:31:49.117426 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:49.117394 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:31:50.554747 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:50.554710 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:31:51.695636 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:51.695605 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:31:52.763459 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:52.763392 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:31:53.924515 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:53.924464 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:31:55.355012 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:55.354966 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:31:56.331554 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.331515 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:31:56.332990 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.332971 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-xqjvm"] Sep 08 18:31:56.333293 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.333282 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02e7edc5-9a02-4eb1-b05e-f7f10f461da6" containerName="pull" Sep 08 18:31:56.333334 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.333295 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="02e7edc5-9a02-4eb1-b05e-f7f10f461da6" containerName="pull" Sep 08 18:31:56.333334 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.333310 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85ddbfff-5b0c-4d54-97d0-0915bed12562" containerName="util" Sep 08 18:31:56.333334 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.333314 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="85ddbfff-5b0c-4d54-97d0-0915bed12562" containerName="util" Sep 08 18:31:56.333334 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.333321 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c74608dd-db3f-42d1-bd92-9855edc26559" containerName="util" Sep 08 18:31:56.333334 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.333326 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="c74608dd-db3f-42d1-bd92-9855edc26559" containerName="util" Sep 08 18:31:56.333334 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.333334 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02e7edc5-9a02-4eb1-b05e-f7f10f461da6" containerName="util" Sep 08 18:31:56.333504 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.333339 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="02e7edc5-9a02-4eb1-b05e-f7f10f461da6" containerName="util" Sep 08 18:31:56.333504 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.333344 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c74608dd-db3f-42d1-bd92-9855edc26559" containerName="extract" Sep 08 18:31:56.333504 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.333349 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="c74608dd-db3f-42d1-bd92-9855edc26559" containerName="extract" Sep 08 18:31:56.333504 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.333361 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9ebe3c3-e921-4f7e-8724-48bbcdd02e40" containerName="extract" Sep 08 18:31:56.333504 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.333366 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9ebe3c3-e921-4f7e-8724-48bbcdd02e40" containerName="extract" Sep 08 18:31:56.333504 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.333372 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02e7edc5-9a02-4eb1-b05e-f7f10f461da6" containerName="extract" Sep 08 18:31:56.333504 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.333377 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="02e7edc5-9a02-4eb1-b05e-f7f10f461da6" containerName="extract" Sep 08 18:31:56.333504 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.333383 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c74608dd-db3f-42d1-bd92-9855edc26559" containerName="pull" Sep 08 18:31:56.333504 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.333387 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="c74608dd-db3f-42d1-bd92-9855edc26559" containerName="pull" Sep 08 18:31:56.333504 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.333394 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9ebe3c3-e921-4f7e-8724-48bbcdd02e40" containerName="util" Sep 08 18:31:56.333504 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.333398 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9ebe3c3-e921-4f7e-8724-48bbcdd02e40" containerName="util" Sep 08 18:31:56.333504 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.333405 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85ddbfff-5b0c-4d54-97d0-0915bed12562" containerName="pull" Sep 08 18:31:56.333504 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.333409 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="85ddbfff-5b0c-4d54-97d0-0915bed12562" containerName="pull" Sep 08 18:31:56.333504 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.333414 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85ddbfff-5b0c-4d54-97d0-0915bed12562" containerName="extract" Sep 08 18:31:56.333504 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.333418 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="85ddbfff-5b0c-4d54-97d0-0915bed12562" containerName="extract" Sep 08 18:31:56.333504 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.333424 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9ebe3c3-e921-4f7e-8724-48bbcdd02e40" containerName="pull" Sep 08 18:31:56.333504 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.333430 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9ebe3c3-e921-4f7e-8724-48bbcdd02e40" containerName="pull" Sep 08 18:31:56.333504 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.333480 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="02e7edc5-9a02-4eb1-b05e-f7f10f461da6" containerName="extract" Sep 08 18:31:56.333504 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.333488 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="c74608dd-db3f-42d1-bd92-9855edc26559" containerName="extract" Sep 08 18:31:56.333504 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.333493 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="85ddbfff-5b0c-4d54-97d0-0915bed12562" containerName="extract" Sep 08 18:31:56.333504 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.333502 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9ebe3c3-e921-4f7e-8724-48bbcdd02e40" containerName="extract" Sep 08 18:31:56.337885 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.337869 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-xqjvm" Sep 08 18:31:56.339856 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.339826 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Sep 08 18:31:56.340351 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.340332 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Sep 08 18:31:56.340503 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.340479 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 08 18:31:56.340632 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.340544 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 08 18:31:56.340632 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.340551 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-9k5mn\"" Sep 08 18:31:56.348387 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.348368 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-xqjvm"] Sep 08 18:31:56.427282 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.427241 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe663dac-2e58-406f-92f2-9ea9868b662f-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-xqjvm\" (UID: \"fe663dac-2e58-406f-92f2-9ea9868b662f\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-xqjvm" Sep 08 18:31:56.427282 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.427280 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wz9qn\" (UniqueName: \"kubernetes.io/projected/fe663dac-2e58-406f-92f2-9ea9868b662f-kube-api-access-wz9qn\") pod \"kuadrant-console-plugin-fbdd4f445-xqjvm\" (UID: \"fe663dac-2e58-406f-92f2-9ea9868b662f\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-xqjvm" Sep 08 18:31:56.427729 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.427315 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/fe663dac-2e58-406f-92f2-9ea9868b662f-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-xqjvm\" (UID: \"fe663dac-2e58-406f-92f2-9ea9868b662f\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-xqjvm" Sep 08 18:31:56.528794 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.528757 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/fe663dac-2e58-406f-92f2-9ea9868b662f-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-xqjvm\" (UID: \"fe663dac-2e58-406f-92f2-9ea9868b662f\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-xqjvm" Sep 08 18:31:56.528973 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.528867 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe663dac-2e58-406f-92f2-9ea9868b662f-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-xqjvm\" (UID: \"fe663dac-2e58-406f-92f2-9ea9868b662f\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-xqjvm" Sep 08 18:31:56.528973 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.528898 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz9qn\" (UniqueName: \"kubernetes.io/projected/fe663dac-2e58-406f-92f2-9ea9868b662f-kube-api-access-wz9qn\") pod \"kuadrant-console-plugin-fbdd4f445-xqjvm\" (UID: \"fe663dac-2e58-406f-92f2-9ea9868b662f\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-xqjvm" Sep 08 18:31:56.529062 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:31:56.529015 2555 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Sep 08 18:31:56.529098 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:31:56.529082 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe663dac-2e58-406f-92f2-9ea9868b662f-plugin-serving-cert podName:fe663dac-2e58-406f-92f2-9ea9868b662f nodeName:}" failed. No retries permitted until 2026-09-08 18:31:57.029063878 +0000 UTC m=+499.235049253 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/fe663dac-2e58-406f-92f2-9ea9868b662f-plugin-serving-cert") pod "kuadrant-console-plugin-fbdd4f445-xqjvm" (UID: "fe663dac-2e58-406f-92f2-9ea9868b662f") : secret "plugin-serving-cert" not found Sep 08 18:31:56.529800 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.529783 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/fe663dac-2e58-406f-92f2-9ea9868b662f-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-xqjvm\" (UID: \"fe663dac-2e58-406f-92f2-9ea9868b662f\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-xqjvm" Sep 08 18:31:56.539225 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.539200 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wz9qn\" (UniqueName: \"kubernetes.io/projected/fe663dac-2e58-406f-92f2-9ea9868b662f-kube-api-access-wz9qn\") pod \"kuadrant-console-plugin-fbdd4f445-xqjvm\" (UID: \"fe663dac-2e58-406f-92f2-9ea9868b662f\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-xqjvm" Sep 08 18:31:56.943714 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:56.943659 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:31:57.032202 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:57.032165 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe663dac-2e58-406f-92f2-9ea9868b662f-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-xqjvm\" (UID: \"fe663dac-2e58-406f-92f2-9ea9868b662f\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-xqjvm" Sep 08 18:31:57.034671 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:57.034641 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe663dac-2e58-406f-92f2-9ea9868b662f-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-xqjvm\" (UID: \"fe663dac-2e58-406f-92f2-9ea9868b662f\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-xqjvm" Sep 08 18:31:57.246889 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:57.246808 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-xqjvm" Sep 08 18:31:57.518474 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:57.518448 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-xqjvm"] Sep 08 18:31:57.520451 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:31:57.520426 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfe663dac_2e58_406f_92f2_9ea9868b662f.slice/crio-3fc3464243163d75d664352d738e492ecda03f3173196d2dfe5e1f7b717d4eb1 WatchSource:0}: Error finding container 3fc3464243163d75d664352d738e492ecda03f3173196d2dfe5e1f7b717d4eb1: Status 404 returned error can't find the container with id 3fc3464243163d75d664352d738e492ecda03f3173196d2dfe5e1f7b717d4eb1 Sep 08 18:31:57.721710 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:57.721651 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-xqjvm" event={"ID":"fe663dac-2e58-406f-92f2-9ea9868b662f","Type":"ContainerStarted","Data":"3fc3464243163d75d664352d738e492ecda03f3173196d2dfe5e1f7b717d4eb1"} Sep 08 18:31:59.299850 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:59.299820 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:31:59.420377 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:59.420342 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:31:59.705235 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:59.705160 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:31:59.818168 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:31:59.818128 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:32:00.104852 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:00.104814 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:32:00.177207 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:00.177178 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:32:02.758541 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:02.758505 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-xqjvm" event={"ID":"fe663dac-2e58-406f-92f2-9ea9868b662f","Type":"ContainerStarted","Data":"7d19e9ed1cda304959bec6d9fb4344bd03f3da8353a6bbf741fef02ac3e8619d"} Sep 08 18:32:02.777750 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:02.777704 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-xqjvm" podStartSLOduration=2.360553714 podStartE2EDuration="6.77767416s" totalImagesPullingTime="4.417120446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:31:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:32:02.777130833 +0000 UTC m=+504.983116230" watchObservedRunningTime="2026-09-08 18:32:02.77767416 +0000 UTC m=+504.983659559" Sep 08 18:32:09.582119 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:09.582078 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:32:09.675257 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:09.675217 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:32:10.120868 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:10.120837 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:32:10.220133 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:10.220100 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:32:13.899214 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:13.899181 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:32:14.000596 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:14.000558 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:32:16.910598 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:16.910514 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:32:16.978209 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:16.978177 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:32:23.915164 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:23.915132 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:32:24.015697 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:24.015656 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:32:24.316849 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:24.316816 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:32:24.422331 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:24.422294 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:32:24.749629 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:24.749545 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:32:24.857551 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:24.857514 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:32:32.444924 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:32.444837 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:32:32.545081 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:32.545047 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:32:35.427470 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:35.427437 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:32:35.527505 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:35.527466 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:32:37.879292 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:37.879254 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:32:37.936666 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:37.936635 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:32:39.760440 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:39.760404 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:32:39.944214 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:39.944180 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:32:40.949963 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:40.949924 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:32:42.310480 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:42.310444 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:32:42.406126 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:42.406095 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:32:42.519522 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:42.519494 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:32:42.728774 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:42.728671 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:32:42.750723 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:42.750674 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:32:42.750898 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:42.750776 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-prfq4" Sep 08 18:32:42.753066 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:42.753034 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Sep 08 18:32:42.755083 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:42.755051 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:32:42.833026 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:42.832987 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b033b298-b4fe-4dbc-bf2e-119bd2723454-config-file\") pod \"limitador-limitador-84994f7ffc-prfq4\" (UID: \"b033b298-b4fe-4dbc-bf2e-119bd2723454\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-prfq4" Sep 08 18:32:42.833205 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:42.833060 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wj4mc\" (UniqueName: \"kubernetes.io/projected/b033b298-b4fe-4dbc-bf2e-119bd2723454-kube-api-access-wj4mc\") pod \"limitador-limitador-84994f7ffc-prfq4\" (UID: \"b033b298-b4fe-4dbc-bf2e-119bd2723454\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-prfq4" Sep 08 18:32:42.933878 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:42.933838 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b033b298-b4fe-4dbc-bf2e-119bd2723454-config-file\") pod \"limitador-limitador-84994f7ffc-prfq4\" (UID: \"b033b298-b4fe-4dbc-bf2e-119bd2723454\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-prfq4" Sep 08 18:32:42.934069 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:42.933939 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj4mc\" (UniqueName: \"kubernetes.io/projected/b033b298-b4fe-4dbc-bf2e-119bd2723454-kube-api-access-wj4mc\") pod \"limitador-limitador-84994f7ffc-prfq4\" (UID: \"b033b298-b4fe-4dbc-bf2e-119bd2723454\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-prfq4" Sep 08 18:32:42.934484 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:42.934466 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b033b298-b4fe-4dbc-bf2e-119bd2723454-config-file\") pod \"limitador-limitador-84994f7ffc-prfq4\" (UID: \"b033b298-b4fe-4dbc-bf2e-119bd2723454\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-prfq4" Sep 08 18:32:42.942160 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:42.942130 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj4mc\" (UniqueName: \"kubernetes.io/projected/b033b298-b4fe-4dbc-bf2e-119bd2723454-kube-api-access-wj4mc\") pod \"limitador-limitador-84994f7ffc-prfq4\" (UID: \"b033b298-b4fe-4dbc-bf2e-119bd2723454\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-prfq4" Sep 08 18:32:43.061411 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:43.061370 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-prfq4" Sep 08 18:32:43.159893 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:43.159863 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:32:43.230234 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:43.230201 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:32:43.231063 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:43.231040 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-4rb87"] Sep 08 18:32:43.234794 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:43.234773 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-4rb87" Sep 08 18:32:43.238421 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:43.237119 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-lvhv4\"" Sep 08 18:32:43.245144 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:43.245110 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-4rb87"] Sep 08 18:32:43.333584 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:43.333553 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:32:43.334806 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:32:43.334778 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb033b298_b4fe_4dbc_bf2e_119bd2723454.slice/crio-740020a42c179f79c1bf89f269390cdab8585732232c9dd3b84355c78dec6282 WatchSource:0}: Error finding container 740020a42c179f79c1bf89f269390cdab8585732232c9dd3b84355c78dec6282: Status 404 returned error can't find the container with id 740020a42c179f79c1bf89f269390cdab8585732232c9dd3b84355c78dec6282 Sep 08 18:32:43.339529 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:43.339507 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwdnh\" (UniqueName: \"kubernetes.io/projected/1b2fbdd8-b489-462a-9afb-2438656f4167-kube-api-access-xwdnh\") pod \"authorino-59c974bd48-4rb87\" (UID: \"1b2fbdd8-b489-462a-9afb-2438656f4167\") " pod="kuadrant-system/authorino-59c974bd48-4rb87" Sep 08 18:32:43.440257 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:43.440213 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwdnh\" (UniqueName: \"kubernetes.io/projected/1b2fbdd8-b489-462a-9afb-2438656f4167-kube-api-access-xwdnh\") pod \"authorino-59c974bd48-4rb87\" (UID: \"1b2fbdd8-b489-462a-9afb-2438656f4167\") " pod="kuadrant-system/authorino-59c974bd48-4rb87" Sep 08 18:32:43.448439 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:43.448412 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwdnh\" (UniqueName: \"kubernetes.io/projected/1b2fbdd8-b489-462a-9afb-2438656f4167-kube-api-access-xwdnh\") pod \"authorino-59c974bd48-4rb87\" (UID: \"1b2fbdd8-b489-462a-9afb-2438656f4167\") " pod="kuadrant-system/authorino-59c974bd48-4rb87" Sep 08 18:32:43.550231 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:43.550194 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-4rb87" Sep 08 18:32:43.820902 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:43.820872 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-4rb87"] Sep 08 18:32:43.822387 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:32:43.822363 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1b2fbdd8_b489_462a_9afb_2438656f4167.slice/crio-4783503be6b4eee076b0ce5463faf6fadc31b28c51bf75e600a3f5cc4573bb7b WatchSource:0}: Error finding container 4783503be6b4eee076b0ce5463faf6fadc31b28c51bf75e600a3f5cc4573bb7b: Status 404 returned error can't find the container with id 4783503be6b4eee076b0ce5463faf6fadc31b28c51bf75e600a3f5cc4573bb7b Sep 08 18:32:43.978087 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:43.977992 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-4rb87" event={"ID":"1b2fbdd8-b489-462a-9afb-2438656f4167","Type":"ContainerStarted","Data":"4783503be6b4eee076b0ce5463faf6fadc31b28c51bf75e600a3f5cc4573bb7b"} Sep 08 18:32:43.979217 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:43.979190 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-prfq4" event={"ID":"b033b298-b4fe-4dbc-bf2e-119bd2723454","Type":"ContainerStarted","Data":"740020a42c179f79c1bf89f269390cdab8585732232c9dd3b84355c78dec6282"} Sep 08 18:32:47.000115 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:47.000074 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-4rb87" event={"ID":"1b2fbdd8-b489-462a-9afb-2438656f4167","Type":"ContainerStarted","Data":"d9af05376503ddb67f350df456bdee62c46eb43674bc8d26ba8dd11bd4424a22"} Sep 08 18:32:47.001629 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:47.001602 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-prfq4" event={"ID":"b033b298-b4fe-4dbc-bf2e-119bd2723454","Type":"ContainerStarted","Data":"2516cd3e7f540c51093ca1c24b17d8ee8824e234ef597f8fbdb37afc97cc2270"} Sep 08 18:32:47.001756 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:47.001668 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-prfq4" Sep 08 18:32:47.019402 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:47.019351 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-4rb87" podStartSLOduration=1.6035975850000002 podStartE2EDuration="4.019336038s" totalImagesPullingTime="2.415738453s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:32:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:32:47.016573894 +0000 UTC m=+549.222559292" watchObservedRunningTime="2026-09-08 18:32:47.019336038 +0000 UTC m=+549.225321436" Sep 08 18:32:47.050527 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:47.050474 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-prfq4" podStartSLOduration=2.154170326 podStartE2EDuration="5.050456982s" totalImagesPullingTime="2.896286656s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:32:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:32:47.049118689 +0000 UTC m=+549.255104101" watchObservedRunningTime="2026-09-08 18:32:47.050456982 +0000 UTC m=+549.256442381" Sep 08 18:32:58.007662 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:32:58.007632 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-prfq4" Sep 08 18:33:03.516994 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:03.516959 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:33:03.656973 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:03.656935 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:33:04.005777 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:04.005744 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:33:15.746739 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:15.746703 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:33:15.970888 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:15.970852 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-xlbz4"] Sep 08 18:33:15.977134 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:15.977117 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-xlbz4" Sep 08 18:33:15.981730 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:15.981704 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-xlbz4"] Sep 08 18:33:16.033358 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:16.033282 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztzk7\" (UniqueName: \"kubernetes.io/projected/4a995698-1920-4582-b039-09dd6d79ed27-kube-api-access-ztzk7\") pod \"authorino-557f876f69-xlbz4\" (UID: \"4a995698-1920-4582-b039-09dd6d79ed27\") " pod="kuadrant-system/authorino-557f876f69-xlbz4" Sep 08 18:33:16.134016 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:16.133984 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztzk7\" (UniqueName: \"kubernetes.io/projected/4a995698-1920-4582-b039-09dd6d79ed27-kube-api-access-ztzk7\") pod \"authorino-557f876f69-xlbz4\" (UID: \"4a995698-1920-4582-b039-09dd6d79ed27\") " pod="kuadrant-system/authorino-557f876f69-xlbz4" Sep 08 18:33:16.142805 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:16.142773 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztzk7\" (UniqueName: \"kubernetes.io/projected/4a995698-1920-4582-b039-09dd6d79ed27-kube-api-access-ztzk7\") pod \"authorino-557f876f69-xlbz4\" (UID: \"4a995698-1920-4582-b039-09dd6d79ed27\") " pod="kuadrant-system/authorino-557f876f69-xlbz4" Sep 08 18:33:16.212714 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:16.212672 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-xlbz4"] Sep 08 18:33:16.212948 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:16.212935 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-xlbz4" Sep 08 18:33:16.235532 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:16.235507 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b67f55dd5-6h8ft"] Sep 08 18:33:16.240098 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:16.240080 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b67f55dd5-6h8ft" Sep 08 18:33:16.246099 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:16.246058 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b67f55dd5-6h8ft"] Sep 08 18:33:16.298330 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:16.298304 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:33:16.336082 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:16.336053 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqqgp\" (UniqueName: \"kubernetes.io/projected/d4d0d432-37c3-4942-9072-86300a81e0a6-kube-api-access-tqqgp\") pod \"authorino-b67f55dd5-6h8ft\" (UID: \"d4d0d432-37c3-4942-9072-86300a81e0a6\") " pod="kuadrant-system/authorino-b67f55dd5-6h8ft" Sep 08 18:33:16.340010 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:16.339989 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b67f55dd5-6h8ft"] Sep 08 18:33:16.340270 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:33:16.340250 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-tqqgp], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-b67f55dd5-6h8ft" podUID="d4d0d432-37c3-4942-9072-86300a81e0a6" Sep 08 18:33:16.436962 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:16.436933 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqqgp\" (UniqueName: \"kubernetes.io/projected/d4d0d432-37c3-4942-9072-86300a81e0a6-kube-api-access-tqqgp\") pod \"authorino-b67f55dd5-6h8ft\" (UID: \"d4d0d432-37c3-4942-9072-86300a81e0a6\") " pod="kuadrant-system/authorino-b67f55dd5-6h8ft" Sep 08 18:33:16.444358 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:16.444338 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqqgp\" (UniqueName: \"kubernetes.io/projected/d4d0d432-37c3-4942-9072-86300a81e0a6-kube-api-access-tqqgp\") pod \"authorino-b67f55dd5-6h8ft\" (UID: \"d4d0d432-37c3-4942-9072-86300a81e0a6\") " pod="kuadrant-system/authorino-b67f55dd5-6h8ft" Sep 08 18:33:16.489349 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:16.489328 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-xlbz4"] Sep 08 18:33:16.491337 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:33:16.491310 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4a995698_1920_4582_b039_09dd6d79ed27.slice/crio-c4524d5d70e8527239cff18f65674a67a78a0ff26142bc0ab4d2a5c6982aa2d7 WatchSource:0}: Error finding container c4524d5d70e8527239cff18f65674a67a78a0ff26142bc0ab4d2a5c6982aa2d7: Status 404 returned error can't find the container with id c4524d5d70e8527239cff18f65674a67a78a0ff26142bc0ab4d2a5c6982aa2d7 Sep 08 18:33:17.177071 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:17.177026 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-xlbz4" event={"ID":"4a995698-1920-4582-b039-09dd6d79ed27","Type":"ContainerStarted","Data":"becea0a0dc0509a281130b4040e884d45c1f587347f88ee25fad9873464b36b5"} Sep 08 18:33:17.177071 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:17.177056 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b67f55dd5-6h8ft" Sep 08 18:33:17.177547 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:17.177081 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-xlbz4" event={"ID":"4a995698-1920-4582-b039-09dd6d79ed27","Type":"ContainerStarted","Data":"c4524d5d70e8527239cff18f65674a67a78a0ff26142bc0ab4d2a5c6982aa2d7"} Sep 08 18:33:17.177547 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:17.177141 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-557f876f69-xlbz4" podUID="4a995698-1920-4582-b039-09dd6d79ed27" containerName="authorino" containerID="cri-o://becea0a0dc0509a281130b4040e884d45c1f587347f88ee25fad9873464b36b5" gracePeriod=30 Sep 08 18:33:17.208089 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:17.208040 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-557f876f69-xlbz4" podStartSLOduration=1.600713083 podStartE2EDuration="2.208025696s" totalImagesPullingTime="607.312613ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:33:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:33:17.205883599 +0000 UTC m=+579.411868998" watchObservedRunningTime="2026-09-08 18:33:17.208025696 +0000 UTC m=+579.414011092" Sep 08 18:33:17.248074 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:17.248052 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b67f55dd5-6h8ft" Sep 08 18:33:17.345301 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:17.345121 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqqgp\" (UniqueName: \"kubernetes.io/projected/d4d0d432-37c3-4942-9072-86300a81e0a6-kube-api-access-tqqgp\") pod \"d4d0d432-37c3-4942-9072-86300a81e0a6\" (UID: \"d4d0d432-37c3-4942-9072-86300a81e0a6\") " Sep 08 18:33:17.349880 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:17.349819 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4d0d432-37c3-4942-9072-86300a81e0a6-kube-api-access-tqqgp" (OuterVolumeSpecName: "kube-api-access-tqqgp") pod "d4d0d432-37c3-4942-9072-86300a81e0a6" (UID: "d4d0d432-37c3-4942-9072-86300a81e0a6"). InnerVolumeSpecName "kube-api-access-tqqgp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:33:17.422530 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:17.422505 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-xlbz4" Sep 08 18:33:17.446766 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:17.446742 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqqgp\" (UniqueName: \"kubernetes.io/projected/d4d0d432-37c3-4942-9072-86300a81e0a6-kube-api-access-tqqgp\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:33:17.547638 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:17.547602 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ztzk7\" (UniqueName: \"kubernetes.io/projected/4a995698-1920-4582-b039-09dd6d79ed27-kube-api-access-ztzk7\") pod \"4a995698-1920-4582-b039-09dd6d79ed27\" (UID: \"4a995698-1920-4582-b039-09dd6d79ed27\") " Sep 08 18:33:17.549979 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:17.549955 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a995698-1920-4582-b039-09dd6d79ed27-kube-api-access-ztzk7" (OuterVolumeSpecName: "kube-api-access-ztzk7") pod "4a995698-1920-4582-b039-09dd6d79ed27" (UID: "4a995698-1920-4582-b039-09dd6d79ed27"). InnerVolumeSpecName "kube-api-access-ztzk7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:33:17.648301 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:17.648193 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ztzk7\" (UniqueName: \"kubernetes.io/projected/4a995698-1920-4582-b039-09dd6d79ed27-kube-api-access-ztzk7\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:33:18.184137 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:18.184100 2555 generic.go:356] "Generic (PLEG): container finished" podID="4a995698-1920-4582-b039-09dd6d79ed27" containerID="becea0a0dc0509a281130b4040e884d45c1f587347f88ee25fad9873464b36b5" exitCode=2 Sep 08 18:33:18.184550 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:18.184144 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-xlbz4" Sep 08 18:33:18.184550 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:18.184187 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-xlbz4" event={"ID":"4a995698-1920-4582-b039-09dd6d79ed27","Type":"ContainerDied","Data":"becea0a0dc0509a281130b4040e884d45c1f587347f88ee25fad9873464b36b5"} Sep 08 18:33:18.184550 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:18.184223 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-xlbz4" event={"ID":"4a995698-1920-4582-b039-09dd6d79ed27","Type":"ContainerDied","Data":"c4524d5d70e8527239cff18f65674a67a78a0ff26142bc0ab4d2a5c6982aa2d7"} Sep 08 18:33:18.184550 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:18.184239 2555 scope.go:117] "RemoveContainer" containerID="becea0a0dc0509a281130b4040e884d45c1f587347f88ee25fad9873464b36b5" Sep 08 18:33:18.184550 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:18.184351 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b67f55dd5-6h8ft" Sep 08 18:33:18.194275 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:18.194253 2555 scope.go:117] "RemoveContainer" containerID="becea0a0dc0509a281130b4040e884d45c1f587347f88ee25fad9873464b36b5" Sep 08 18:33:18.194576 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:33:18.194555 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"becea0a0dc0509a281130b4040e884d45c1f587347f88ee25fad9873464b36b5\": container with ID starting with becea0a0dc0509a281130b4040e884d45c1f587347f88ee25fad9873464b36b5 not found: ID does not exist" containerID="becea0a0dc0509a281130b4040e884d45c1f587347f88ee25fad9873464b36b5" Sep 08 18:33:18.194645 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:18.194583 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"becea0a0dc0509a281130b4040e884d45c1f587347f88ee25fad9873464b36b5"} err="failed to get container status \"becea0a0dc0509a281130b4040e884d45c1f587347f88ee25fad9873464b36b5\": rpc error: code = NotFound desc = could not find container \"becea0a0dc0509a281130b4040e884d45c1f587347f88ee25fad9873464b36b5\": container with ID starting with becea0a0dc0509a281130b4040e884d45c1f587347f88ee25fad9873464b36b5 not found: ID does not exist" Sep 08 18:33:18.212107 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:18.212078 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b67f55dd5-6h8ft"] Sep 08 18:33:18.215754 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:18.215725 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b67f55dd5-6h8ft"] Sep 08 18:33:18.226833 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:18.226780 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-xlbz4"] Sep 08 18:33:18.231323 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:18.231299 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-xlbz4"] Sep 08 18:33:18.324370 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:18.324337 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a995698-1920-4582-b039-09dd6d79ed27" path="/var/lib/kubelet/pods/4a995698-1920-4582-b039-09dd6d79ed27/volumes" Sep 08 18:33:18.324662 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:18.324649 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4d0d432-37c3-4942-9072-86300a81e0a6" path="/var/lib/kubelet/pods/d4d0d432-37c3-4942-9072-86300a81e0a6/volumes" Sep 08 18:33:25.523237 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:25.523193 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-4rb87"] Sep 08 18:33:25.523680 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:25.523416 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-4rb87" podUID="1b2fbdd8-b489-462a-9afb-2438656f4167" containerName="authorino" containerID="cri-o://d9af05376503ddb67f350df456bdee62c46eb43674bc8d26ba8dd11bd4424a22" gracePeriod=30 Sep 08 18:33:25.774198 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:25.774139 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-4rb87" Sep 08 18:33:25.920333 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:25.920302 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xwdnh\" (UniqueName: \"kubernetes.io/projected/1b2fbdd8-b489-462a-9afb-2438656f4167-kube-api-access-xwdnh\") pod \"1b2fbdd8-b489-462a-9afb-2438656f4167\" (UID: \"1b2fbdd8-b489-462a-9afb-2438656f4167\") " Sep 08 18:33:25.922504 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:25.922479 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b2fbdd8-b489-462a-9afb-2438656f4167-kube-api-access-xwdnh" (OuterVolumeSpecName: "kube-api-access-xwdnh") pod "1b2fbdd8-b489-462a-9afb-2438656f4167" (UID: "1b2fbdd8-b489-462a-9afb-2438656f4167"). InnerVolumeSpecName "kube-api-access-xwdnh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:33:26.021303 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:26.021263 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xwdnh\" (UniqueName: \"kubernetes.io/projected/1b2fbdd8-b489-462a-9afb-2438656f4167-kube-api-access-xwdnh\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:33:26.229130 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:26.229093 2555 generic.go:356] "Generic (PLEG): container finished" podID="1b2fbdd8-b489-462a-9afb-2438656f4167" containerID="d9af05376503ddb67f350df456bdee62c46eb43674bc8d26ba8dd11bd4424a22" exitCode=0 Sep 08 18:33:26.229315 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:26.229135 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-4rb87" Sep 08 18:33:26.229315 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:26.229162 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-4rb87" event={"ID":"1b2fbdd8-b489-462a-9afb-2438656f4167","Type":"ContainerDied","Data":"d9af05376503ddb67f350df456bdee62c46eb43674bc8d26ba8dd11bd4424a22"} Sep 08 18:33:26.229315 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:26.229194 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-4rb87" event={"ID":"1b2fbdd8-b489-462a-9afb-2438656f4167","Type":"ContainerDied","Data":"4783503be6b4eee076b0ce5463faf6fadc31b28c51bf75e600a3f5cc4573bb7b"} Sep 08 18:33:26.229315 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:26.229211 2555 scope.go:117] "RemoveContainer" containerID="d9af05376503ddb67f350df456bdee62c46eb43674bc8d26ba8dd11bd4424a22" Sep 08 18:33:26.241020 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:26.240996 2555 scope.go:117] "RemoveContainer" containerID="d9af05376503ddb67f350df456bdee62c46eb43674bc8d26ba8dd11bd4424a22" Sep 08 18:33:26.241577 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:33:26.241553 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9af05376503ddb67f350df456bdee62c46eb43674bc8d26ba8dd11bd4424a22\": container with ID starting with d9af05376503ddb67f350df456bdee62c46eb43674bc8d26ba8dd11bd4424a22 not found: ID does not exist" containerID="d9af05376503ddb67f350df456bdee62c46eb43674bc8d26ba8dd11bd4424a22" Sep 08 18:33:26.241669 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:26.241586 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9af05376503ddb67f350df456bdee62c46eb43674bc8d26ba8dd11bd4424a22"} err="failed to get container status \"d9af05376503ddb67f350df456bdee62c46eb43674bc8d26ba8dd11bd4424a22\": rpc error: code = NotFound desc = could not find container \"d9af05376503ddb67f350df456bdee62c46eb43674bc8d26ba8dd11bd4424a22\": container with ID starting with d9af05376503ddb67f350df456bdee62c46eb43674bc8d26ba8dd11bd4424a22 not found: ID does not exist" Sep 08 18:33:26.250953 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:26.250926 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-4rb87"] Sep 08 18:33:26.255002 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:26.254979 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-4rb87"] Sep 08 18:33:26.324196 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:26.324167 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b2fbdd8-b489-462a-9afb-2438656f4167" path="/var/lib/kubelet/pods/1b2fbdd8-b489-462a-9afb-2438656f4167/volumes" Sep 08 18:33:30.178759 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:30.178713 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:33:30.292855 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:30.292811 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:33:30.396246 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:30.396205 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:33:30.646777 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:30.646744 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:33:33.296603 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:33.296548 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:33:33.399112 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:33.399084 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:33:38.271713 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:38.271662 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9n28f_911c61b8-766b-4ae8-b046-784e22e85889/console-operator/1.log" Sep 08 18:33:38.272136 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:33:38.272123 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9n28f_911c61b8-766b-4ae8-b046-784e22e85889/console-operator/1.log" Sep 08 18:34:04.951850 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:04.951766 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:34:06.075327 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:06.075292 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:34:06.330179 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:06.330085 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:34:06.411390 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:06.411356 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:34:06.484395 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:06.484358 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:34:06.585445 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:06.585340 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:34:06.792583 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:06.792555 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:34:06.893481 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:06.893393 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:34:06.938929 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:06.938892 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:34:16.926733 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:16.926613 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:34:17.033926 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:17.033888 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:34:21.980877 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:21.980821 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:34:22.352415 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:22.352381 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:34:35.142074 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:35.142038 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:34:35.825368 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:35.825336 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:34:43.295879 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:43.295816 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:34:43.397079 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:43.397046 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:34:51.423958 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:51.423921 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw"] Sep 08 18:34:51.424368 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:51.424344 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a995698-1920-4582-b039-09dd6d79ed27" containerName="authorino" Sep 08 18:34:51.424368 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:51.424356 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a995698-1920-4582-b039-09dd6d79ed27" containerName="authorino" Sep 08 18:34:51.424447 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:51.424378 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b2fbdd8-b489-462a-9afb-2438656f4167" containerName="authorino" Sep 08 18:34:51.424447 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:51.424384 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b2fbdd8-b489-462a-9afb-2438656f4167" containerName="authorino" Sep 08 18:34:51.424447 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:51.424440 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a995698-1920-4582-b039-09dd6d79ed27" containerName="authorino" Sep 08 18:34:51.424558 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:51.424450 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b2fbdd8-b489-462a-9afb-2438656f4167" containerName="authorino" Sep 08 18:34:51.427630 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:51.427614 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw" Sep 08 18:34:51.429844 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:51.429820 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 08 18:34:51.430309 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:51.430288 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 08 18:34:51.430408 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:51.430306 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\"" Sep 08 18:34:51.430408 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:51.430336 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-2gzvd\"" Sep 08 18:34:51.437789 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:51.437767 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw"] Sep 08 18:34:51.481347 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:51.481315 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd2423f3-b698-4319-84bd-34fc45ce7e7b-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw\" (UID: \"dd2423f3-b698-4319-84bd-34fc45ce7e7b\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw" Sep 08 18:34:51.481534 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:51.481371 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rdxp\" (UniqueName: \"kubernetes.io/projected/dd2423f3-b698-4319-84bd-34fc45ce7e7b-kube-api-access-7rdxp\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw\" (UID: \"dd2423f3-b698-4319-84bd-34fc45ce7e7b\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw" Sep 08 18:34:51.481534 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:51.481394 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd2423f3-b698-4319-84bd-34fc45ce7e7b-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw\" (UID: \"dd2423f3-b698-4319-84bd-34fc45ce7e7b\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw" Sep 08 18:34:51.481705 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:51.481523 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd2423f3-b698-4319-84bd-34fc45ce7e7b-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw\" (UID: \"dd2423f3-b698-4319-84bd-34fc45ce7e7b\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw" Sep 08 18:34:51.481705 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:51.481581 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd2423f3-b698-4319-84bd-34fc45ce7e7b-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw\" (UID: \"dd2423f3-b698-4319-84bd-34fc45ce7e7b\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw" Sep 08 18:34:51.481705 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:51.481613 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd2423f3-b698-4319-84bd-34fc45ce7e7b-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw\" (UID: \"dd2423f3-b698-4319-84bd-34fc45ce7e7b\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw" Sep 08 18:34:51.521503 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:51.521468 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:34:51.582605 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:51.582563 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd2423f3-b698-4319-84bd-34fc45ce7e7b-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw\" (UID: \"dd2423f3-b698-4319-84bd-34fc45ce7e7b\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw" Sep 08 18:34:51.582605 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:51.582605 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd2423f3-b698-4319-84bd-34fc45ce7e7b-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw\" (UID: \"dd2423f3-b698-4319-84bd-34fc45ce7e7b\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw" Sep 08 18:34:51.582907 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:51.582627 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd2423f3-b698-4319-84bd-34fc45ce7e7b-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw\" (UID: \"dd2423f3-b698-4319-84bd-34fc45ce7e7b\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw" Sep 08 18:34:51.582907 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:51.582678 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd2423f3-b698-4319-84bd-34fc45ce7e7b-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw\" (UID: \"dd2423f3-b698-4319-84bd-34fc45ce7e7b\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw" Sep 08 18:34:51.582907 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:51.582798 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rdxp\" (UniqueName: \"kubernetes.io/projected/dd2423f3-b698-4319-84bd-34fc45ce7e7b-kube-api-access-7rdxp\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw\" (UID: \"dd2423f3-b698-4319-84bd-34fc45ce7e7b\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw" Sep 08 18:34:51.582907 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:51.582824 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd2423f3-b698-4319-84bd-34fc45ce7e7b-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw\" (UID: \"dd2423f3-b698-4319-84bd-34fc45ce7e7b\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw" Sep 08 18:34:51.583133 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:51.583104 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd2423f3-b698-4319-84bd-34fc45ce7e7b-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw\" (UID: \"dd2423f3-b698-4319-84bd-34fc45ce7e7b\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw" Sep 08 18:34:51.583198 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:51.583125 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd2423f3-b698-4319-84bd-34fc45ce7e7b-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw\" (UID: \"dd2423f3-b698-4319-84bd-34fc45ce7e7b\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw" Sep 08 18:34:51.583198 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:51.583187 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd2423f3-b698-4319-84bd-34fc45ce7e7b-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw\" (UID: \"dd2423f3-b698-4319-84bd-34fc45ce7e7b\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw" Sep 08 18:34:51.584934 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:51.584917 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd2423f3-b698-4319-84bd-34fc45ce7e7b-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw\" (UID: \"dd2423f3-b698-4319-84bd-34fc45ce7e7b\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw" Sep 08 18:34:51.585351 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:51.585329 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd2423f3-b698-4319-84bd-34fc45ce7e7b-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw\" (UID: \"dd2423f3-b698-4319-84bd-34fc45ce7e7b\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw" Sep 08 18:34:51.591945 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:51.591921 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rdxp\" (UniqueName: \"kubernetes.io/projected/dd2423f3-b698-4319-84bd-34fc45ce7e7b-kube-api-access-7rdxp\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw\" (UID: \"dd2423f3-b698-4319-84bd-34fc45ce7e7b\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw" Sep 08 18:34:51.738452 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:51.738348 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw" Sep 08 18:34:52.012250 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:52.012219 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw"] Sep 08 18:34:52.013593 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:34:52.013546 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd2423f3_b698_4319_84bd_34fc45ce7e7b.slice/crio-cf43b1e20c2226c69529d67b57d42bfff3081b1520676049c365dac70ace85fa WatchSource:0}: Error finding container cf43b1e20c2226c69529d67b57d42bfff3081b1520676049c365dac70ace85fa: Status 404 returned error can't find the container with id cf43b1e20c2226c69529d67b57d42bfff3081b1520676049c365dac70ace85fa Sep 08 18:34:52.015118 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:52.015103 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 08 18:34:52.425177 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:52.425139 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:34:52.701724 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:52.701602 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw" event={"ID":"dd2423f3-b698-4319-84bd-34fc45ce7e7b","Type":"ContainerStarted","Data":"cf43b1e20c2226c69529d67b57d42bfff3081b1520676049c365dac70ace85fa"} Sep 08 18:34:53.709112 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:53.709069 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw" event={"ID":"dd2423f3-b698-4319-84bd-34fc45ce7e7b","Type":"ContainerStarted","Data":"c0a6a94b91cad88b01c4dfb50e8ef4df1d5b72c69194169ad37db60dd93a8091"} Sep 08 18:34:53.728823 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:34:53.728761 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw" podStartSLOduration=1.690619812 podStartE2EDuration="2.728745714s" totalImagesPullingTime="1.038125902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:34:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:34:53.725857062 +0000 UTC m=+675.931842462" watchObservedRunningTime="2026-09-08 18:34:53.728745714 +0000 UTC m=+675.934731110" Sep 08 18:35:01.739149 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:01.739111 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw" Sep 08 18:35:01.739547 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:01.739195 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw" Sep 08 18:35:01.751664 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:01.751638 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw" Sep 08 18:35:01.764110 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:01.764083 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw" Sep 08 18:35:04.775730 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:04.775676 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:35:05.333118 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:05.333067 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:35:06.292577 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:06.292545 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:35:15.847607 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:15.847576 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:35:17.048287 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:17.048251 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:35:19.942076 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:19.942038 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:35:21.428317 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:21.428279 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:35:27.986397 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:27.986359 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:35:28.784643 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:28.784605 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:35:29.744053 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:29.743966 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:35:32.573508 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:32.573477 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:35:39.808457 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:39.808425 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd"] Sep 08 18:35:39.812025 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:39.812005 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd" Sep 08 18:35:39.814424 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:39.814404 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Sep 08 18:35:39.822662 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:39.822641 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd"] Sep 08 18:35:39.834149 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:39.834120 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:35:39.896571 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:39.896547 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca6972fc-daa1-4d54-a149-ab81b4187e5c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd\" (UID: \"ca6972fc-daa1-4d54-a149-ab81b4187e5c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd" Sep 08 18:35:39.896732 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:39.896584 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca6972fc-daa1-4d54-a149-ab81b4187e5c-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd\" (UID: \"ca6972fc-daa1-4d54-a149-ab81b4187e5c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd" Sep 08 18:35:39.896732 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:39.896654 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca6972fc-daa1-4d54-a149-ab81b4187e5c-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd\" (UID: \"ca6972fc-daa1-4d54-a149-ab81b4187e5c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd" Sep 08 18:35:39.896732 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:39.896667 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca6972fc-daa1-4d54-a149-ab81b4187e5c-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd\" (UID: \"ca6972fc-daa1-4d54-a149-ab81b4187e5c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd" Sep 08 18:35:39.896732 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:39.896716 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bffh\" (UniqueName: \"kubernetes.io/projected/ca6972fc-daa1-4d54-a149-ab81b4187e5c-kube-api-access-8bffh\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd\" (UID: \"ca6972fc-daa1-4d54-a149-ab81b4187e5c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd" Sep 08 18:35:39.896874 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:39.896743 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca6972fc-daa1-4d54-a149-ab81b4187e5c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd\" (UID: \"ca6972fc-daa1-4d54-a149-ab81b4187e5c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd" Sep 08 18:35:39.997615 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:39.997567 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca6972fc-daa1-4d54-a149-ab81b4187e5c-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd\" (UID: \"ca6972fc-daa1-4d54-a149-ab81b4187e5c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd" Sep 08 18:35:39.997615 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:39.997614 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca6972fc-daa1-4d54-a149-ab81b4187e5c-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd\" (UID: \"ca6972fc-daa1-4d54-a149-ab81b4187e5c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd" Sep 08 18:35:39.997843 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:39.997636 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bffh\" (UniqueName: \"kubernetes.io/projected/ca6972fc-daa1-4d54-a149-ab81b4187e5c-kube-api-access-8bffh\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd\" (UID: \"ca6972fc-daa1-4d54-a149-ab81b4187e5c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd" Sep 08 18:35:39.997843 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:39.997660 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca6972fc-daa1-4d54-a149-ab81b4187e5c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd\" (UID: \"ca6972fc-daa1-4d54-a149-ab81b4187e5c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd" Sep 08 18:35:39.997843 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:39.997713 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca6972fc-daa1-4d54-a149-ab81b4187e5c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd\" (UID: \"ca6972fc-daa1-4d54-a149-ab81b4187e5c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd" Sep 08 18:35:39.997843 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:39.997736 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca6972fc-daa1-4d54-a149-ab81b4187e5c-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd\" (UID: \"ca6972fc-daa1-4d54-a149-ab81b4187e5c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd" Sep 08 18:35:39.998105 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:39.998084 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca6972fc-daa1-4d54-a149-ab81b4187e5c-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd\" (UID: \"ca6972fc-daa1-4d54-a149-ab81b4187e5c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd" Sep 08 18:35:39.998173 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:39.998147 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca6972fc-daa1-4d54-a149-ab81b4187e5c-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd\" (UID: \"ca6972fc-daa1-4d54-a149-ab81b4187e5c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd" Sep 08 18:35:39.998214 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:39.998174 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca6972fc-daa1-4d54-a149-ab81b4187e5c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd\" (UID: \"ca6972fc-daa1-4d54-a149-ab81b4187e5c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd" Sep 08 18:35:40.000068 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:40.000047 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca6972fc-daa1-4d54-a149-ab81b4187e5c-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd\" (UID: \"ca6972fc-daa1-4d54-a149-ab81b4187e5c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd" Sep 08 18:35:40.000264 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:40.000244 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca6972fc-daa1-4d54-a149-ab81b4187e5c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd\" (UID: \"ca6972fc-daa1-4d54-a149-ab81b4187e5c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd" Sep 08 18:35:40.006290 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:40.006271 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bffh\" (UniqueName: \"kubernetes.io/projected/ca6972fc-daa1-4d54-a149-ab81b4187e5c-kube-api-access-8bffh\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd\" (UID: \"ca6972fc-daa1-4d54-a149-ab81b4187e5c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd" Sep 08 18:35:40.121556 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:40.121459 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd" Sep 08 18:35:40.397739 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:40.397717 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd"] Sep 08 18:35:40.399217 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:35:40.399197 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podca6972fc_daa1_4d54_a149_ab81b4187e5c.slice/crio-0f03c468947b78f565c5a620ad8b109571c17d06543d4be3694e505e49261d4f WatchSource:0}: Error finding container 0f03c468947b78f565c5a620ad8b109571c17d06543d4be3694e505e49261d4f: Status 404 returned error can't find the container with id 0f03c468947b78f565c5a620ad8b109571c17d06543d4be3694e505e49261d4f Sep 08 18:35:40.961659 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:40.961627 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd" event={"ID":"ca6972fc-daa1-4d54-a149-ab81b4187e5c","Type":"ContainerStarted","Data":"5bc7b9e8c5189023638c9819b3d93eeef731507dfbbe60aeb17e12107ff96705"} Sep 08 18:35:40.962054 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:40.961668 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd" event={"ID":"ca6972fc-daa1-4d54-a149-ab81b4187e5c","Type":"ContainerStarted","Data":"0f03c468947b78f565c5a620ad8b109571c17d06543d4be3694e505e49261d4f"} Sep 08 18:35:40.989660 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:40.989614 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd" podStartSLOduration=1.8062189709999998 podStartE2EDuration="1.989598497s" totalImagesPullingTime="183.379526ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:35:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:35:40.989315958 +0000 UTC m=+723.195301356" watchObservedRunningTime="2026-09-08 18:35:40.989598497 +0000 UTC m=+723.195583895" Sep 08 18:35:41.835712 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:41.835655 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:35:50.122368 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:50.122333 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd" Sep 08 18:35:50.122864 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:50.122380 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd" Sep 08 18:35:50.134875 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:50.134852 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd" Sep 08 18:35:51.025785 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:35:51.025757 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd" Sep 08 18:36:13.193845 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:13.193798 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:36:29.132214 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:29.132180 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d7947bbf4-z8l5z"] Sep 08 18:36:29.135633 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:29.135613 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d7947bbf4-z8l5z" Sep 08 18:36:29.138326 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:29.138300 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-lvhv4\"" Sep 08 18:36:29.138326 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:29.138317 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Sep 08 18:36:29.142425 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:29.142400 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d7947bbf4-z8l5z"] Sep 08 18:36:29.233490 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:29.233457 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gq6t\" (UniqueName: \"kubernetes.io/projected/c4f2cdc0-591a-4a3e-8ad1-d4010e3203de-kube-api-access-8gq6t\") pod \"authorino-6d7947bbf4-z8l5z\" (UID: \"c4f2cdc0-591a-4a3e-8ad1-d4010e3203de\") " pod="kuadrant-system/authorino-6d7947bbf4-z8l5z" Sep 08 18:36:29.233490 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:29.233495 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c4f2cdc0-591a-4a3e-8ad1-d4010e3203de-tls-cert\") pod \"authorino-6d7947bbf4-z8l5z\" (UID: \"c4f2cdc0-591a-4a3e-8ad1-d4010e3203de\") " pod="kuadrant-system/authorino-6d7947bbf4-z8l5z" Sep 08 18:36:29.334157 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:29.334123 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gq6t\" (UniqueName: \"kubernetes.io/projected/c4f2cdc0-591a-4a3e-8ad1-d4010e3203de-kube-api-access-8gq6t\") pod \"authorino-6d7947bbf4-z8l5z\" (UID: \"c4f2cdc0-591a-4a3e-8ad1-d4010e3203de\") " pod="kuadrant-system/authorino-6d7947bbf4-z8l5z" Sep 08 18:36:29.334157 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:29.334158 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c4f2cdc0-591a-4a3e-8ad1-d4010e3203de-tls-cert\") pod \"authorino-6d7947bbf4-z8l5z\" (UID: \"c4f2cdc0-591a-4a3e-8ad1-d4010e3203de\") " pod="kuadrant-system/authorino-6d7947bbf4-z8l5z" Sep 08 18:36:29.336655 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:29.336627 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c4f2cdc0-591a-4a3e-8ad1-d4010e3203de-tls-cert\") pod \"authorino-6d7947bbf4-z8l5z\" (UID: \"c4f2cdc0-591a-4a3e-8ad1-d4010e3203de\") " pod="kuadrant-system/authorino-6d7947bbf4-z8l5z" Sep 08 18:36:29.343292 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:29.343265 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gq6t\" (UniqueName: \"kubernetes.io/projected/c4f2cdc0-591a-4a3e-8ad1-d4010e3203de-kube-api-access-8gq6t\") pod \"authorino-6d7947bbf4-z8l5z\" (UID: \"c4f2cdc0-591a-4a3e-8ad1-d4010e3203de\") " pod="kuadrant-system/authorino-6d7947bbf4-z8l5z" Sep 08 18:36:29.446539 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:29.446452 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d7947bbf4-z8l5z" Sep 08 18:36:29.715979 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:29.715956 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d7947bbf4-z8l5z"] Sep 08 18:36:29.718173 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:36:29.718142 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc4f2cdc0_591a_4a3e_8ad1_d4010e3203de.slice/crio-d16615cf37d317670d1373a76bd3f47e5e10be7736ff9daf3f152d552710bc56 WatchSource:0}: Error finding container d16615cf37d317670d1373a76bd3f47e5e10be7736ff9daf3f152d552710bc56: Status 404 returned error can't find the container with id d16615cf37d317670d1373a76bd3f47e5e10be7736ff9daf3f152d552710bc56 Sep 08 18:36:30.232471 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:30.232438 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d7947bbf4-z8l5z" event={"ID":"c4f2cdc0-591a-4a3e-8ad1-d4010e3203de","Type":"ContainerStarted","Data":"d16615cf37d317670d1373a76bd3f47e5e10be7736ff9daf3f152d552710bc56"} Sep 08 18:36:31.239274 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:31.239235 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d7947bbf4-z8l5z" event={"ID":"c4f2cdc0-591a-4a3e-8ad1-d4010e3203de","Type":"ContainerStarted","Data":"0b481bb143e3d79d61bd691e679ff0dfb5717393b75ea321d88151d84087a3c1"} Sep 08 18:36:31.253562 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:31.253500 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6d7947bbf4-z8l5z" podStartSLOduration=1.474974997 podStartE2EDuration="2.253482976s" totalImagesPullingTime="778.507979ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:36:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:36:31.251899095 +0000 UTC m=+773.457884503" watchObservedRunningTime="2026-09-08 18:36:31.253482976 +0000 UTC m=+773.459468372" Sep 08 18:36:31.794229 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:31.794196 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:36:31.902648 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:31.902618 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:36:33.024087 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:33.024055 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:36:33.117755 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:33.117724 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:36:48.511722 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:48.511641 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:36:48.668392 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:48.668347 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:36:48.836700 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:48.836646 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:36:48.925997 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:48.925954 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:36:49.905398 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:49.905341 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:36:50.005360 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:50.005320 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:36:50.379268 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:50.379234 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:36:52.036753 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:52.036714 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:36:52.359024 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:52.358995 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:36:52.450966 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:52.450939 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:36:53.208584 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:53.208556 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7587-62a779"] Sep 08 18:36:53.212234 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:53.212219 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7587-62a779" Sep 08 18:36:53.214447 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:53.214423 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-mhbkz\"" Sep 08 18:36:53.232492 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:53.232469 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7587-62a779"] Sep 08 18:36:53.346273 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:53.346243 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc89m\" (UniqueName: \"kubernetes.io/projected/52c97008-7dae-4693-bbee-f5b54a1f9a56-kube-api-access-wc89m\") pod \"test-curl-7587-62a779\" (UID: \"52c97008-7dae-4693-bbee-f5b54a1f9a56\") " pod="opendatahub/test-curl-7587-62a779" Sep 08 18:36:53.447651 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:53.447625 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc89m\" (UniqueName: \"kubernetes.io/projected/52c97008-7dae-4693-bbee-f5b54a1f9a56-kube-api-access-wc89m\") pod \"test-curl-7587-62a779\" (UID: \"52c97008-7dae-4693-bbee-f5b54a1f9a56\") " pod="opendatahub/test-curl-7587-62a779" Sep 08 18:36:53.455284 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:53.455257 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc89m\" (UniqueName: \"kubernetes.io/projected/52c97008-7dae-4693-bbee-f5b54a1f9a56-kube-api-access-wc89m\") pod \"test-curl-7587-62a779\" (UID: \"52c97008-7dae-4693-bbee-f5b54a1f9a56\") " pod="opendatahub/test-curl-7587-62a779" Sep 08 18:36:53.521057 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:53.520989 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7587-62a779" Sep 08 18:36:53.651905 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:53.651841 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:36:53.747244 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:53.747214 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:36:53.791348 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:53.791329 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7587-62a779"] Sep 08 18:36:53.793152 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:36:53.793130 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod52c97008_7dae_4693_bbee_f5b54a1f9a56.slice/crio-76c437f76fa34ff5c426d593df6ae9dd471cfd120e44c76a72038afff6e4ae14 WatchSource:0}: Error finding container 76c437f76fa34ff5c426d593df6ae9dd471cfd120e44c76a72038afff6e4ae14: Status 404 returned error can't find the container with id 76c437f76fa34ff5c426d593df6ae9dd471cfd120e44c76a72038afff6e4ae14 Sep 08 18:36:53.983057 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:53.983034 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:36:54.151863 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:54.151789 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:36:54.227004 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:54.226982 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:36:54.367203 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:54.367169 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7587-62a779" event={"ID":"52c97008-7dae-4693-bbee-f5b54a1f9a56","Type":"ContainerStarted","Data":"76c437f76fa34ff5c426d593df6ae9dd471cfd120e44c76a72038afff6e4ae14"} Sep 08 18:36:56.379742 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:56.379706 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7587-62a779" event={"ID":"52c97008-7dae-4693-bbee-f5b54a1f9a56","Type":"ContainerStarted","Data":"e23df04c80cbf257c7e8fcbe82d81005399e38a1cb6bff8d0400714be8773e01"} Sep 08 18:36:56.392890 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:56.392837 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7587-62a779" podStartSLOduration=1.477173197 podStartE2EDuration="3.39282338s" totalImagesPullingTime="1.915650183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:36:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:36:56.392773109 +0000 UTC m=+798.598758502" watchObservedRunningTime="2026-09-08 18:36:56.39282338 +0000 UTC m=+798.598808776" Sep 08 18:36:56.501655 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:56.501628 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:36:56.574828 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:56.574793 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:36:56.736210 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:56.736144 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:36:56.986542 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:56.986464 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7587-62a779"] Sep 08 18:36:56.996816 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:56.996789 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7587-62a779"] Sep 08 18:36:57.052535 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:57.052506 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:36:57.357071 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:57.357037 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:36:57.385487 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:36:57.385451 2555 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7587-62a779" container="test-curl-7587-62a779" Sep 08 18:36:57.516115 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:57.516081 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:36:57.835357 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:57.835327 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:36:58.322241 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:58.322213 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:36:58.390640 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:58.390612 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7587-62a779" podUID="52c97008-7dae-4693-bbee-f5b54a1f9a56" containerName="test-curl-7587-62a779" containerID="cri-o://e23df04c80cbf257c7e8fcbe82d81005399e38a1cb6bff8d0400714be8773e01" gracePeriod=2 Sep 08 18:36:59.854979 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:59.854904 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:36:59.944139 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:36:59.944104 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:00.580021 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:00.580002 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7587-62a779" Sep 08 18:37:00.606999 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:00.606977 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wc89m\" (UniqueName: \"kubernetes.io/projected/52c97008-7dae-4693-bbee-f5b54a1f9a56-kube-api-access-wc89m\") pod \"52c97008-7dae-4693-bbee-f5b54a1f9a56\" (UID: \"52c97008-7dae-4693-bbee-f5b54a1f9a56\") " Sep 08 18:37:00.609047 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:00.609023 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52c97008-7dae-4693-bbee-f5b54a1f9a56-kube-api-access-wc89m" (OuterVolumeSpecName: "kube-api-access-wc89m") pod "52c97008-7dae-4693-bbee-f5b54a1f9a56" (UID: "52c97008-7dae-4693-bbee-f5b54a1f9a56"). InnerVolumeSpecName "kube-api-access-wc89m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:37:00.708049 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:00.707990 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wc89m\" (UniqueName: \"kubernetes.io/projected/52c97008-7dae-4693-bbee-f5b54a1f9a56-kube-api-access-wc89m\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:37:01.051814 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:01.051782 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:01.169002 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:01.168972 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:01.272838 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:01.272808 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:01.410759 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:01.410680 2555 generic.go:356] "Generic (PLEG): container finished" podID="52c97008-7dae-4693-bbee-f5b54a1f9a56" containerID="e23df04c80cbf257c7e8fcbe82d81005399e38a1cb6bff8d0400714be8773e01" exitCode=137 Sep 08 18:37:01.410759 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:01.410743 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7587-62a779" Sep 08 18:37:01.410922 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:01.410792 2555 scope.go:117] "RemoveContainer" containerID="e23df04c80cbf257c7e8fcbe82d81005399e38a1cb6bff8d0400714be8773e01" Sep 08 18:37:01.420032 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:01.420011 2555 scope.go:117] "RemoveContainer" containerID="e23df04c80cbf257c7e8fcbe82d81005399e38a1cb6bff8d0400714be8773e01" Sep 08 18:37:01.420286 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:37:01.420268 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e23df04c80cbf257c7e8fcbe82d81005399e38a1cb6bff8d0400714be8773e01\": container with ID starting with e23df04c80cbf257c7e8fcbe82d81005399e38a1cb6bff8d0400714be8773e01 not found: ID does not exist" containerID="e23df04c80cbf257c7e8fcbe82d81005399e38a1cb6bff8d0400714be8773e01" Sep 08 18:37:01.420345 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:01.420297 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e23df04c80cbf257c7e8fcbe82d81005399e38a1cb6bff8d0400714be8773e01"} err="failed to get container status \"e23df04c80cbf257c7e8fcbe82d81005399e38a1cb6bff8d0400714be8773e01\": rpc error: code = NotFound desc = could not find container \"e23df04c80cbf257c7e8fcbe82d81005399e38a1cb6bff8d0400714be8773e01\": container with ID starting with e23df04c80cbf257c7e8fcbe82d81005399e38a1cb6bff8d0400714be8773e01 not found: ID does not exist" Sep 08 18:37:01.425353 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:37:01.425323 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7587-62a779\" is forbidden: User \"system:node:ip-10-0-139-112.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-112.ec2.internal' and this object" podUID="52c97008-7dae-4693-bbee-f5b54a1f9a56" pod="opendatahub/test-curl-7587-62a779" Sep 08 18:37:01.470791 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:01.470751 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:01.554187 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:01.554159 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:01.655666 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:01.655637 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:01.754344 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:01.754271 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:01.774633 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:01.774610 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-abd9a747-54b9ffd4d5-m9gsq"] Sep 08 18:37:01.775003 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:01.774990 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52c97008-7dae-4693-bbee-f5b54a1f9a56" containerName="test-curl-7587-62a779" Sep 08 18:37:01.775043 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:01.775004 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="52c97008-7dae-4693-bbee-f5b54a1f9a56" containerName="test-curl-7587-62a779" Sep 08 18:37:01.775084 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:01.775075 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="52c97008-7dae-4693-bbee-f5b54a1f9a56" containerName="test-curl-7587-62a779" Sep 08 18:37:01.779349 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:01.779334 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-abd9a747-54b9ffd4d5-m9gsq" Sep 08 18:37:01.782253 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:37:01.782225 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7587-62a779\" is forbidden: User \"system:node:ip-10-0-139-112.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-112.ec2.internal' and this object" podUID="52c97008-7dae-4693-bbee-f5b54a1f9a56" pod="opendatahub/test-curl-7587-62a779" Sep 08 18:37:01.782646 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:01.782628 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-abd9a747-dockercfg-6cjwj\"" Sep 08 18:37:01.782894 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:01.782880 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-abd9a747\"" Sep 08 18:37:01.794721 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:01.794675 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-abd9a747-54b9ffd4d5-m9gsq"] Sep 08 18:37:01.816388 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:01.816363 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q72rb\" (UniqueName: \"kubernetes.io/projected/e39243cb-6eb0-4718-b3a4-3e355f4c0a9c-kube-api-access-q72rb\") pod \"payload-processing-e2e-ait-abd9a747-54b9ffd4d5-m9gsq\" (UID: \"e39243cb-6eb0-4718-b3a4-3e355f4c0a9c\") " pod="openshift-ingress/payload-processing-e2e-ait-abd9a747-54b9ffd4d5-m9gsq" Sep 08 18:37:01.816467 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:01.816405 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e39243cb-6eb0-4718-b3a4-3e355f4c0a9c-plugins-config-volume\") pod \"payload-processing-e2e-ait-abd9a747-54b9ffd4d5-m9gsq\" (UID: \"e39243cb-6eb0-4718-b3a4-3e355f4c0a9c\") " pod="openshift-ingress/payload-processing-e2e-ait-abd9a747-54b9ffd4d5-m9gsq" Sep 08 18:37:01.916844 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:01.916821 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q72rb\" (UniqueName: \"kubernetes.io/projected/e39243cb-6eb0-4718-b3a4-3e355f4c0a9c-kube-api-access-q72rb\") pod \"payload-processing-e2e-ait-abd9a747-54b9ffd4d5-m9gsq\" (UID: \"e39243cb-6eb0-4718-b3a4-3e355f4c0a9c\") " pod="openshift-ingress/payload-processing-e2e-ait-abd9a747-54b9ffd4d5-m9gsq" Sep 08 18:37:01.916931 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:01.916861 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e39243cb-6eb0-4718-b3a4-3e355f4c0a9c-plugins-config-volume\") pod \"payload-processing-e2e-ait-abd9a747-54b9ffd4d5-m9gsq\" (UID: \"e39243cb-6eb0-4718-b3a4-3e355f4c0a9c\") " pod="openshift-ingress/payload-processing-e2e-ait-abd9a747-54b9ffd4d5-m9gsq" Sep 08 18:37:01.917460 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:01.917443 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e39243cb-6eb0-4718-b3a4-3e355f4c0a9c-plugins-config-volume\") pod \"payload-processing-e2e-ait-abd9a747-54b9ffd4d5-m9gsq\" (UID: \"e39243cb-6eb0-4718-b3a4-3e355f4c0a9c\") " pod="openshift-ingress/payload-processing-e2e-ait-abd9a747-54b9ffd4d5-m9gsq" Sep 08 18:37:01.925745 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:01.925726 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q72rb\" (UniqueName: \"kubernetes.io/projected/e39243cb-6eb0-4718-b3a4-3e355f4c0a9c-kube-api-access-q72rb\") pod \"payload-processing-e2e-ait-abd9a747-54b9ffd4d5-m9gsq\" (UID: \"e39243cb-6eb0-4718-b3a4-3e355f4c0a9c\") " pod="openshift-ingress/payload-processing-e2e-ait-abd9a747-54b9ffd4d5-m9gsq" Sep 08 18:37:02.089789 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:02.089767 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-abd9a747-54b9ffd4d5-m9gsq" Sep 08 18:37:02.324698 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:02.324667 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52c97008-7dae-4693-bbee-f5b54a1f9a56" path="/var/lib/kubelet/pods/52c97008-7dae-4693-bbee-f5b54a1f9a56/volumes" Sep 08 18:37:02.366012 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:02.365991 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-abd9a747-54b9ffd4d5-m9gsq"] Sep 08 18:37:02.367605 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:37:02.367580 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode39243cb_6eb0_4718_b3a4_3e355f4c0a9c.slice/crio-cff38206bf08c0126fd6f0c7716d9274ee1196accc132aa954114fd91915505f WatchSource:0}: Error finding container cff38206bf08c0126fd6f0c7716d9274ee1196accc132aa954114fd91915505f: Status 404 returned error can't find the container with id cff38206bf08c0126fd6f0c7716d9274ee1196accc132aa954114fd91915505f Sep 08 18:37:02.418311 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:02.418289 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-abd9a747-54b9ffd4d5-m9gsq" event={"ID":"e39243cb-6eb0-4718-b3a4-3e355f4c0a9c","Type":"ContainerStarted","Data":"cff38206bf08c0126fd6f0c7716d9274ee1196accc132aa954114fd91915505f"} Sep 08 18:37:02.767025 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:02.766960 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:02.866898 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:02.866868 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:03.124432 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:03.124404 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:03.243906 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:03.243881 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:03.327641 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:03.327611 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:03.381154 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:03.381080 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-0eaf99-6b47649f65-v6jj6"] Sep 08 18:37:03.384718 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:03.384680 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-0eaf99-6b47649f65-v6jj6" Sep 08 18:37:03.386988 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:03.386964 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-0eaf99-dockercfg-vc82r\"" Sep 08 18:37:03.387109 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:03.387007 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-0eaf99\"" Sep 08 18:37:03.396420 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:03.396399 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-0eaf99-6b47649f65-v6jj6"] Sep 08 18:37:03.428672 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:03.428637 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e9117f7-78a4-4288-bfd9-9dbc3e7c170a-plugins-config-volume\") pod \"payload-processing-e2e-autores-0eaf99-6b47649f65-v6jj6\" (UID: \"3e9117f7-78a4-4288-bfd9-9dbc3e7c170a\") " pod="openshift-ingress/payload-processing-e2e-autores-0eaf99-6b47649f65-v6jj6" Sep 08 18:37:03.428840 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:03.428724 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sn9g\" (UniqueName: \"kubernetes.io/projected/3e9117f7-78a4-4288-bfd9-9dbc3e7c170a-kube-api-access-5sn9g\") pod \"payload-processing-e2e-autores-0eaf99-6b47649f65-v6jj6\" (UID: \"3e9117f7-78a4-4288-bfd9-9dbc3e7c170a\") " pod="openshift-ingress/payload-processing-e2e-autores-0eaf99-6b47649f65-v6jj6" Sep 08 18:37:03.529843 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:03.529812 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e9117f7-78a4-4288-bfd9-9dbc3e7c170a-plugins-config-volume\") pod \"payload-processing-e2e-autores-0eaf99-6b47649f65-v6jj6\" (UID: \"3e9117f7-78a4-4288-bfd9-9dbc3e7c170a\") " pod="openshift-ingress/payload-processing-e2e-autores-0eaf99-6b47649f65-v6jj6" Sep 08 18:37:03.529980 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:03.529901 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sn9g\" (UniqueName: \"kubernetes.io/projected/3e9117f7-78a4-4288-bfd9-9dbc3e7c170a-kube-api-access-5sn9g\") pod \"payload-processing-e2e-autores-0eaf99-6b47649f65-v6jj6\" (UID: \"3e9117f7-78a4-4288-bfd9-9dbc3e7c170a\") " pod="openshift-ingress/payload-processing-e2e-autores-0eaf99-6b47649f65-v6jj6" Sep 08 18:37:03.530626 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:03.530600 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e9117f7-78a4-4288-bfd9-9dbc3e7c170a-plugins-config-volume\") pod \"payload-processing-e2e-autores-0eaf99-6b47649f65-v6jj6\" (UID: \"3e9117f7-78a4-4288-bfd9-9dbc3e7c170a\") " pod="openshift-ingress/payload-processing-e2e-autores-0eaf99-6b47649f65-v6jj6" Sep 08 18:37:03.538106 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:03.538079 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sn9g\" (UniqueName: \"kubernetes.io/projected/3e9117f7-78a4-4288-bfd9-9dbc3e7c170a-kube-api-access-5sn9g\") pod \"payload-processing-e2e-autores-0eaf99-6b47649f65-v6jj6\" (UID: \"3e9117f7-78a4-4288-bfd9-9dbc3e7c170a\") " pod="openshift-ingress/payload-processing-e2e-autores-0eaf99-6b47649f65-v6jj6" Sep 08 18:37:03.698300 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:03.698219 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-0eaf99-6b47649f65-v6jj6" Sep 08 18:37:03.980059 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:03.980030 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-0eaf99-6b47649f65-v6jj6"] Sep 08 18:37:04.169790 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:37:04.169757 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e9117f7_78a4_4288_bfd9_9dbc3e7c170a.slice/crio-2cecbb1b8f2220d3b0bd42de1fdcd4598e32f749fceb65c533f9a83402b9c628 WatchSource:0}: Error finding container 2cecbb1b8f2220d3b0bd42de1fdcd4598e32f749fceb65c533f9a83402b9c628: Status 404 returned error can't find the container with id 2cecbb1b8f2220d3b0bd42de1fdcd4598e32f749fceb65c533f9a83402b9c628 Sep 08 18:37:04.432466 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:04.432432 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-abd9a747-54b9ffd4d5-m9gsq" event={"ID":"e39243cb-6eb0-4718-b3a4-3e355f4c0a9c","Type":"ContainerStarted","Data":"f37ee399a375c1f7954b4dd6ece9bb2c674717a36d183b35196c82e8478873f4"} Sep 08 18:37:04.432640 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:04.432497 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-abd9a747-54b9ffd4d5-m9gsq" Sep 08 18:37:04.433676 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:04.433655 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-0eaf99-6b47649f65-v6jj6" event={"ID":"3e9117f7-78a4-4288-bfd9-9dbc3e7c170a","Type":"ContainerStarted","Data":"2cecbb1b8f2220d3b0bd42de1fdcd4598e32f749fceb65c533f9a83402b9c628"} Sep 08 18:37:04.448666 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:04.448624 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-abd9a747-54b9ffd4d5-m9gsq" podStartSLOduration=1.598259075 podStartE2EDuration="3.448610207s" totalImagesPullingTime="1.850351132s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:37:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:37:04.447335431 +0000 UTC m=+806.653320839" watchObservedRunningTime="2026-09-08 18:37:04.448610207 +0000 UTC m=+806.654595603" Sep 08 18:37:04.529540 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:04.529465 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:37:05.442132 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:05.442096 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-0eaf99-6b47649f65-v6jj6" event={"ID":"3e9117f7-78a4-4288-bfd9-9dbc3e7c170a","Type":"ContainerStarted","Data":"2db6fabbf5866b5c3cbd4dbe95ebce17be463cc76f1aa764e53ba1b4b7a2918d"} Sep 08 18:37:05.442558 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:05.442368 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-0eaf99-6b47649f65-v6jj6" Sep 08 18:37:05.459887 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:05.459842 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-0eaf99-6b47649f65-v6jj6" podStartSLOduration=2.084487966 podStartE2EDuration="2.459829558s" totalImagesPullingTime="375.341592ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:37:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:37:05.458984474 +0000 UTC m=+807.664969871" watchObservedRunningTime="2026-09-08 18:37:05.459829558 +0000 UTC m=+807.665814979" Sep 08 18:37:07.346284 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:07.346254 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:07.502644 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:07.502616 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:07.595851 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:07.595823 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:07.694955 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:07.694885 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:09.642559 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:09.642529 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:09.735520 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:09.735492 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:09.856449 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:09.856422 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:10.401873 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:10.401833 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:12.008052 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:12.008017 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:12.980278 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:12.980249 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:14.517367 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:14.517339 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:15.444636 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:15.444603 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-abd9a747-54b9ffd4d5-m9gsq" Sep 08 18:37:15.938950 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:15.938917 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:16.045060 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:16.045020 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:16.147246 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:16.147216 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:16.277958 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:16.277886 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:16.449236 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:16.449214 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-0eaf99-6b47649f65-v6jj6" Sep 08 18:37:17.307091 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:17.307060 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:17.625286 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:17.625188 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:17.899463 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:17.899381 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw"] Sep 08 18:37:17.905063 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:17.905036 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:37:17.908437 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:17.908417 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 08 18:37:17.908437 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:17.908431 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-15d796-openshift-default-dockercfg-cbxpn\"" Sep 08 18:37:17.915354 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:17.915332 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw"] Sep 08 18:37:17.927989 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:17.927894 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:17.955214 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:17.955180 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed6265b9-7057-49d9-85e6-9b08b0809029-workload-socket\") pod \"e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw\" (UID: \"ed6265b9-7057-49d9-85e6-9b08b0809029\") " pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:37:17.955358 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:17.955245 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed6265b9-7057-49d9-85e6-9b08b0809029-credential-socket\") pod \"e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw\" (UID: \"ed6265b9-7057-49d9-85e6-9b08b0809029\") " pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:37:17.955358 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:17.955264 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed6265b9-7057-49d9-85e6-9b08b0809029-workload-certs\") pod \"e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw\" (UID: \"ed6265b9-7057-49d9-85e6-9b08b0809029\") " pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:37:17.955358 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:17.955285 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed6265b9-7057-49d9-85e6-9b08b0809029-istio-podinfo\") pod \"e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw\" (UID: \"ed6265b9-7057-49d9-85e6-9b08b0809029\") " pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:37:17.955358 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:17.955313 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed6265b9-7057-49d9-85e6-9b08b0809029-istio-envoy\") pod \"e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw\" (UID: \"ed6265b9-7057-49d9-85e6-9b08b0809029\") " pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:37:17.955491 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:17.955402 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed6265b9-7057-49d9-85e6-9b08b0809029-istio-data\") pod \"e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw\" (UID: \"ed6265b9-7057-49d9-85e6-9b08b0809029\") " pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:37:17.955491 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:17.955434 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed6265b9-7057-49d9-85e6-9b08b0809029-istiod-ca-cert\") pod \"e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw\" (UID: \"ed6265b9-7057-49d9-85e6-9b08b0809029\") " pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:37:17.955491 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:17.955454 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x244\" (UniqueName: \"kubernetes.io/projected/ed6265b9-7057-49d9-85e6-9b08b0809029-kube-api-access-8x244\") pod \"e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw\" (UID: \"ed6265b9-7057-49d9-85e6-9b08b0809029\") " pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:37:17.955491 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:17.955481 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed6265b9-7057-49d9-85e6-9b08b0809029-istio-token\") pod \"e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw\" (UID: \"ed6265b9-7057-49d9-85e6-9b08b0809029\") " pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:37:18.056514 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:18.056479 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed6265b9-7057-49d9-85e6-9b08b0809029-istio-token\") pod \"e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw\" (UID: \"ed6265b9-7057-49d9-85e6-9b08b0809029\") " pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:37:18.056790 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:18.056536 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed6265b9-7057-49d9-85e6-9b08b0809029-workload-socket\") pod \"e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw\" (UID: \"ed6265b9-7057-49d9-85e6-9b08b0809029\") " pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:37:18.056790 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:18.056581 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed6265b9-7057-49d9-85e6-9b08b0809029-credential-socket\") pod \"e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw\" (UID: \"ed6265b9-7057-49d9-85e6-9b08b0809029\") " pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:37:18.056790 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:18.056599 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed6265b9-7057-49d9-85e6-9b08b0809029-workload-certs\") pod \"e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw\" (UID: \"ed6265b9-7057-49d9-85e6-9b08b0809029\") " pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:37:18.056790 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:18.056632 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed6265b9-7057-49d9-85e6-9b08b0809029-istio-podinfo\") pod \"e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw\" (UID: \"ed6265b9-7057-49d9-85e6-9b08b0809029\") " pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:37:18.056790 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:18.056662 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed6265b9-7057-49d9-85e6-9b08b0809029-istio-envoy\") pod \"e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw\" (UID: \"ed6265b9-7057-49d9-85e6-9b08b0809029\") " pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:37:18.056790 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:18.056750 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed6265b9-7057-49d9-85e6-9b08b0809029-istio-data\") pod \"e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw\" (UID: \"ed6265b9-7057-49d9-85e6-9b08b0809029\") " pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:37:18.056790 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:18.056778 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed6265b9-7057-49d9-85e6-9b08b0809029-istiod-ca-cert\") pod \"e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw\" (UID: \"ed6265b9-7057-49d9-85e6-9b08b0809029\") " pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:37:18.057178 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:18.056798 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x244\" (UniqueName: \"kubernetes.io/projected/ed6265b9-7057-49d9-85e6-9b08b0809029-kube-api-access-8x244\") pod \"e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw\" (UID: \"ed6265b9-7057-49d9-85e6-9b08b0809029\") " pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:37:18.057178 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:18.057022 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed6265b9-7057-49d9-85e6-9b08b0809029-credential-socket\") pod \"e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw\" (UID: \"ed6265b9-7057-49d9-85e6-9b08b0809029\") " pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:37:18.057178 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:18.057047 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed6265b9-7057-49d9-85e6-9b08b0809029-workload-socket\") pod \"e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw\" (UID: \"ed6265b9-7057-49d9-85e6-9b08b0809029\") " pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:37:18.057322 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:18.057215 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed6265b9-7057-49d9-85e6-9b08b0809029-istio-data\") pod \"e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw\" (UID: \"ed6265b9-7057-49d9-85e6-9b08b0809029\") " pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:37:18.057322 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:18.057307 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed6265b9-7057-49d9-85e6-9b08b0809029-workload-certs\") pod \"e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw\" (UID: \"ed6265b9-7057-49d9-85e6-9b08b0809029\") " pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:37:18.057533 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:18.057497 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed6265b9-7057-49d9-85e6-9b08b0809029-istiod-ca-cert\") pod \"e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw\" (UID: \"ed6265b9-7057-49d9-85e6-9b08b0809029\") " pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:37:18.059093 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:18.059069 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed6265b9-7057-49d9-85e6-9b08b0809029-istio-envoy\") pod \"e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw\" (UID: \"ed6265b9-7057-49d9-85e6-9b08b0809029\") " pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:37:18.059249 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:18.059232 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed6265b9-7057-49d9-85e6-9b08b0809029-istio-podinfo\") pod \"e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw\" (UID: \"ed6265b9-7057-49d9-85e6-9b08b0809029\") " pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:37:18.064174 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:18.064147 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed6265b9-7057-49d9-85e6-9b08b0809029-istio-token\") pod \"e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw\" (UID: \"ed6265b9-7057-49d9-85e6-9b08b0809029\") " pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:37:18.064413 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:18.064395 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x244\" (UniqueName: \"kubernetes.io/projected/ed6265b9-7057-49d9-85e6-9b08b0809029-kube-api-access-8x244\") pod \"e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw\" (UID: \"ed6265b9-7057-49d9-85e6-9b08b0809029\") " pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:37:18.175168 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:18.175069 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:18.216344 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:18.216308 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:37:18.494094 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:18.494068 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw"] Sep 08 18:37:18.496146 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:37:18.496115 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded6265b9_7057_49d9_85e6_9b08b0809029.slice/crio-91af66a560460b08f6c99988a5fdd00e12301d04d94323572d354b2f5acb24e0 WatchSource:0}: Error finding container 91af66a560460b08f6c99988a5fdd00e12301d04d94323572d354b2f5acb24e0: Status 404 returned error can't find the container with id 91af66a560460b08f6c99988a5fdd00e12301d04d94323572d354b2f5acb24e0 Sep 08 18:37:18.515862 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:18.515827 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" event={"ID":"ed6265b9-7057-49d9-85e6-9b08b0809029","Type":"ContainerStarted","Data":"91af66a560460b08f6c99988a5fdd00e12301d04d94323572d354b2f5acb24e0"} Sep 08 18:37:18.827040 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:18.827008 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:18.927382 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:18.927339 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:20.131575 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:20.131540 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:37:21.185669 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:21.185631 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 18:37:21.185966 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:21.185721 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 18:37:21.185966 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:21.185752 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 18:37:21.537300 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:21.537273 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" event={"ID":"ed6265b9-7057-49d9-85e6-9b08b0809029","Type":"ContainerStarted","Data":"f8692b6822a42ac57622d7f993cc0fe6293ade303beda66bb8ed2dacf82d713f"} Sep 08 18:37:21.555556 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:21.555499 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" podStartSLOduration=1.868406572 podStartE2EDuration="4.555483797s" totalImagesPullingTime="2.687077225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:37:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:37:21.554337731 +0000 UTC m=+823.760323126" watchObservedRunningTime="2026-09-08 18:37:21.555483797 +0000 UTC m=+823.761469223" Sep 08 18:37:22.070032 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:22.070003 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:22.172091 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:22.172063 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:22.216446 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:22.216416 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:37:22.221412 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:22.221390 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:37:22.249655 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:22.249623 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:22.543301 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:22.543273 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:37:22.544170 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:22.544148 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:37:23.640180 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:23.640142 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:23.722936 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:23.722903 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:24.125352 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:24.125321 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:24.924380 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:24.924343 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:25.024543 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:25.024511 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:25.126963 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:25.126924 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:25.347784 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:25.347750 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-abd9a747-rnwkm"] Sep 08 18:37:25.352050 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:25.352022 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-abd9a747-rnwkm" Sep 08 18:37:25.354089 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:25.354067 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 08 18:37:25.354258 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:25.354240 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g6q8h\"" Sep 08 18:37:25.359281 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:25.359258 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-abd9a747-rnwkm"] Sep 08 18:37:25.380034 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:25.380000 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:25.428738 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:25.428679 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97bc5cea-58f0-4a7e-9b43-06ac862fc49e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-abd9a747-rnwkm\" (UID: \"97bc5cea-58f0-4a7e-9b43-06ac862fc49e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-abd9a747-rnwkm" Sep 08 18:37:25.486111 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:25.486065 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:25.489135 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:25.489113 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-15d796-668bc7c5d-6m7d9"] Sep 08 18:37:25.492900 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:25.492874 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-15d796-668bc7c5d-6m7d9" Sep 08 18:37:25.495291 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:25.495270 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-15d796-dockercfg-fnfp2\"" Sep 08 18:37:25.495468 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:25.495451 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-15d796\"" Sep 08 18:37:25.504732 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:25.504703 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-15d796-668bc7c5d-6m7d9"] Sep 08 18:37:25.514351 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:25.514325 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-15d796-5c8bd9fb99-2zv2v"] Sep 08 18:37:25.517723 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:25.517706 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-15d796-5c8bd9fb99-2zv2v" Sep 08 18:37:25.529252 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:25.529230 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97bc5cea-58f0-4a7e-9b43-06ac862fc49e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-abd9a747-rnwkm\" (UID: \"97bc5cea-58f0-4a7e-9b43-06ac862fc49e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-abd9a747-rnwkm" Sep 08 18:37:25.529908 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:25.529888 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97bc5cea-58f0-4a7e-9b43-06ac862fc49e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-abd9a747-rnwkm\" (UID: \"97bc5cea-58f0-4a7e-9b43-06ac862fc49e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-abd9a747-rnwkm" Sep 08 18:37:25.532834 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:25.532816 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-15d796-5c8bd9fb99-2zv2v"] Sep 08 18:37:25.630869 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:25.630767 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhgsg\" (UniqueName: \"kubernetes.io/projected/241061eb-fa90-45a1-ae52-13730bf5c646-kube-api-access-nhgsg\") pod \"payload-pre-processing-e2e-inf-b-15d796-668bc7c5d-6m7d9\" (UID: \"241061eb-fa90-45a1-ae52-13730bf5c646\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-15d796-668bc7c5d-6m7d9" Sep 08 18:37:25.630869 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:25.630813 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsl2b\" (UniqueName: \"kubernetes.io/projected/4822efcb-f15f-4857-b882-774b0baec527-kube-api-access-hsl2b\") pod \"payload-processing-e2e-inf-b-15d796-5c8bd9fb99-2zv2v\" (UID: \"4822efcb-f15f-4857-b882-774b0baec527\") " pod="openshift-ingress/payload-processing-e2e-inf-b-15d796-5c8bd9fb99-2zv2v" Sep 08 18:37:25.631058 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:25.630913 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4822efcb-f15f-4857-b882-774b0baec527-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-15d796-5c8bd9fb99-2zv2v\" (UID: \"4822efcb-f15f-4857-b882-774b0baec527\") " pod="openshift-ingress/payload-processing-e2e-inf-b-15d796-5c8bd9fb99-2zv2v" Sep 08 18:37:25.631058 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:25.630955 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/241061eb-fa90-45a1-ae52-13730bf5c646-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-15d796-668bc7c5d-6m7d9\" (UID: \"241061eb-fa90-45a1-ae52-13730bf5c646\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-15d796-668bc7c5d-6m7d9" Sep 08 18:37:25.664755 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:25.664724 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-abd9a747-rnwkm" Sep 08 18:37:25.733347 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:25.732611 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4822efcb-f15f-4857-b882-774b0baec527-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-15d796-5c8bd9fb99-2zv2v\" (UID: \"4822efcb-f15f-4857-b882-774b0baec527\") " pod="openshift-ingress/payload-processing-e2e-inf-b-15d796-5c8bd9fb99-2zv2v" Sep 08 18:37:25.733347 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:25.732701 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/241061eb-fa90-45a1-ae52-13730bf5c646-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-15d796-668bc7c5d-6m7d9\" (UID: \"241061eb-fa90-45a1-ae52-13730bf5c646\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-15d796-668bc7c5d-6m7d9" Sep 08 18:37:25.733347 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:25.732827 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhgsg\" (UniqueName: \"kubernetes.io/projected/241061eb-fa90-45a1-ae52-13730bf5c646-kube-api-access-nhgsg\") pod \"payload-pre-processing-e2e-inf-b-15d796-668bc7c5d-6m7d9\" (UID: \"241061eb-fa90-45a1-ae52-13730bf5c646\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-15d796-668bc7c5d-6m7d9" Sep 08 18:37:25.733347 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:25.732867 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsl2b\" (UniqueName: \"kubernetes.io/projected/4822efcb-f15f-4857-b882-774b0baec527-kube-api-access-hsl2b\") pod \"payload-processing-e2e-inf-b-15d796-5c8bd9fb99-2zv2v\" (UID: \"4822efcb-f15f-4857-b882-774b0baec527\") " pod="openshift-ingress/payload-processing-e2e-inf-b-15d796-5c8bd9fb99-2zv2v" Sep 08 18:37:25.733669 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:25.733584 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4822efcb-f15f-4857-b882-774b0baec527-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-15d796-5c8bd9fb99-2zv2v\" (UID: \"4822efcb-f15f-4857-b882-774b0baec527\") " pod="openshift-ingress/payload-processing-e2e-inf-b-15d796-5c8bd9fb99-2zv2v" Sep 08 18:37:25.733923 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:25.733894 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/241061eb-fa90-45a1-ae52-13730bf5c646-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-15d796-668bc7c5d-6m7d9\" (UID: \"241061eb-fa90-45a1-ae52-13730bf5c646\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-15d796-668bc7c5d-6m7d9" Sep 08 18:37:25.741337 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:25.741311 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhgsg\" (UniqueName: \"kubernetes.io/projected/241061eb-fa90-45a1-ae52-13730bf5c646-kube-api-access-nhgsg\") pod \"payload-pre-processing-e2e-inf-b-15d796-668bc7c5d-6m7d9\" (UID: \"241061eb-fa90-45a1-ae52-13730bf5c646\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-15d796-668bc7c5d-6m7d9" Sep 08 18:37:25.741539 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:25.741512 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsl2b\" (UniqueName: \"kubernetes.io/projected/4822efcb-f15f-4857-b882-774b0baec527-kube-api-access-hsl2b\") pod \"payload-processing-e2e-inf-b-15d796-5c8bd9fb99-2zv2v\" (UID: \"4822efcb-f15f-4857-b882-774b0baec527\") " pod="openshift-ingress/payload-processing-e2e-inf-b-15d796-5c8bd9fb99-2zv2v" Sep 08 18:37:25.761054 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:25.761015 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:25.804225 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:25.804192 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-15d796-668bc7c5d-6m7d9" Sep 08 18:37:25.828625 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:25.828537 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-15d796-5c8bd9fb99-2zv2v" Sep 08 18:37:25.939522 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:25.939499 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-abd9a747-rnwkm"] Sep 08 18:37:25.941013 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:37:25.940982 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97bc5cea_58f0_4a7e_9b43_06ac862fc49e.slice/crio-76dd9e61ff12c9e9f32b065f2806452b16839aec89c39aa9199b44980723af7c WatchSource:0}: Error finding container 76dd9e61ff12c9e9f32b065f2806452b16839aec89c39aa9199b44980723af7c: Status 404 returned error can't find the container with id 76dd9e61ff12c9e9f32b065f2806452b16839aec89c39aa9199b44980723af7c Sep 08 18:37:26.078326 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:26.078300 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-15d796-668bc7c5d-6m7d9"] Sep 08 18:37:26.079122 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:37:26.079098 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod241061eb_fa90_45a1_ae52_13730bf5c646.slice/crio-64ef50e258e298af3150d4dc4ebdf576ec080883a4c347dd8b975ffab4d4b9e0 WatchSource:0}: Error finding container 64ef50e258e298af3150d4dc4ebdf576ec080883a4c347dd8b975ffab4d4b9e0: Status 404 returned error can't find the container with id 64ef50e258e298af3150d4dc4ebdf576ec080883a4c347dd8b975ffab4d4b9e0 Sep 08 18:37:26.109302 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:26.109278 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-15d796-5c8bd9fb99-2zv2v"] Sep 08 18:37:26.112793 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:37:26.112771 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4822efcb_f15f_4857_b882_774b0baec527.slice/crio-4848194a8915c507deeb8806f55974114a9e40b94698bded0fe1383a319489c9 WatchSource:0}: Error finding container 4848194a8915c507deeb8806f55974114a9e40b94698bded0fe1383a319489c9: Status 404 returned error can't find the container with id 4848194a8915c507deeb8806f55974114a9e40b94698bded0fe1383a319489c9 Sep 08 18:37:26.390644 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:26.390610 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:26.568082 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:26.568046 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-15d796-5c8bd9fb99-2zv2v" event={"ID":"4822efcb-f15f-4857-b882-774b0baec527","Type":"ContainerStarted","Data":"442291376e5bd98cc82f1abebe1e7731665244be47d11ef881e803356c571a79"} Sep 08 18:37:26.568291 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:26.568103 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-15d796-5c8bd9fb99-2zv2v" event={"ID":"4822efcb-f15f-4857-b882-774b0baec527","Type":"ContainerStarted","Data":"4848194a8915c507deeb8806f55974114a9e40b94698bded0fe1383a319489c9"} Sep 08 18:37:26.568291 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:26.568161 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-15d796-5c8bd9fb99-2zv2v" Sep 08 18:37:26.569579 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:26.569560 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-15d796-668bc7c5d-6m7d9" event={"ID":"241061eb-fa90-45a1-ae52-13730bf5c646","Type":"ContainerStarted","Data":"9cca1fbfdcea09d33163af2fc49eb250038b3e786a9861ede59906d77c5c0b19"} Sep 08 18:37:26.569711 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:26.569597 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-15d796-668bc7c5d-6m7d9" event={"ID":"241061eb-fa90-45a1-ae52-13730bf5c646","Type":"ContainerStarted","Data":"64ef50e258e298af3150d4dc4ebdf576ec080883a4c347dd8b975ffab4d4b9e0"} Sep 08 18:37:26.569711 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:26.569619 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-15d796-668bc7c5d-6m7d9" Sep 08 18:37:26.570736 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:26.570718 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-abd9a747-rnwkm" event={"ID":"97bc5cea-58f0-4a7e-9b43-06ac862fc49e","Type":"ContainerStarted","Data":"76dd9e61ff12c9e9f32b065f2806452b16839aec89c39aa9199b44980723af7c"} Sep 08 18:37:26.589437 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:26.589383 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-15d796-5c8bd9fb99-2zv2v" podStartSLOduration=1.589369298 podStartE2EDuration="1.589369298s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:37:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:37:26.587780844 +0000 UTC m=+828.793766241" watchObservedRunningTime="2026-09-08 18:37:26.589369298 +0000 UTC m=+828.795354695" Sep 08 18:37:26.605287 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:26.605242 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-15d796-668bc7c5d-6m7d9" podStartSLOduration=1.60522951 podStartE2EDuration="1.60522951s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:37:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:37:26.602806223 +0000 UTC m=+828.808791622" watchObservedRunningTime="2026-09-08 18:37:26.60522951 +0000 UTC m=+828.811214907" Sep 08 18:37:26.634367 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:26.634341 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:27.276478 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:27.276443 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:30.595983 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:30.595949 2555 generic.go:356] "Generic (PLEG): container finished" podID="97bc5cea-58f0-4a7e-9b43-06ac862fc49e" containerID="6e1005d8628e72ff4a1566be45b23c3bfd17a702009f6a2b97b7911a2d013d1a" exitCode=0 Sep 08 18:37:30.596347 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:30.596040 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-abd9a747-rnwkm" event={"ID":"97bc5cea-58f0-4a7e-9b43-06ac862fc49e","Type":"ContainerDied","Data":"6e1005d8628e72ff4a1566be45b23c3bfd17a702009f6a2b97b7911a2d013d1a"} Sep 08 18:37:31.741869 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:31.741845 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-abd9a747-rnwkm" Sep 08 18:37:31.891739 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:31.891637 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97bc5cea-58f0-4a7e-9b43-06ac862fc49e-maas-api-service-ca\") pod \"97bc5cea-58f0-4a7e-9b43-06ac862fc49e\" (UID: \"97bc5cea-58f0-4a7e-9b43-06ac862fc49e\") " Sep 08 18:37:31.892036 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:31.892012 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97bc5cea-58f0-4a7e-9b43-06ac862fc49e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "97bc5cea-58f0-4a7e-9b43-06ac862fc49e" (UID: "97bc5cea-58f0-4a7e-9b43-06ac862fc49e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:37:31.993453 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:31.993405 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97bc5cea-58f0-4a7e-9b43-06ac862fc49e-maas-api-service-ca\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:37:32.614908 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:32.614809 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-abd9a747-rnwkm" event={"ID":"97bc5cea-58f0-4a7e-9b43-06ac862fc49e","Type":"ContainerDied","Data":"76dd9e61ff12c9e9f32b065f2806452b16839aec89c39aa9199b44980723af7c"} Sep 08 18:37:32.615176 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:32.615159 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="76dd9e61ff12c9e9f32b065f2806452b16839aec89c39aa9199b44980723af7c" Sep 08 18:37:32.615369 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:32.615351 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-abd9a747-rnwkm" Sep 08 18:37:37.578356 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:37.578317 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-15d796-668bc7c5d-6m7d9" Sep 08 18:37:37.578648 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:37.578389 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-15d796-5c8bd9fb99-2zv2v" Sep 08 18:37:39.231332 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:39.231293 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:37:40.661803 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:40.661732 2555 generic.go:356] "Generic (PLEG): container finished" podID="ed6265b9-7057-49d9-85e6-9b08b0809029" containerID="f8692b6822a42ac57622d7f993cc0fe6293ade303beda66bb8ed2dacf82d713f" exitCode=0 Sep 08 18:37:40.662159 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:40.661804 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" event={"ID":"ed6265b9-7057-49d9-85e6-9b08b0809029","Type":"ContainerDied","Data":"f8692b6822a42ac57622d7f993cc0fe6293ade303beda66bb8ed2dacf82d713f"} Sep 08 18:37:40.662240 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:40.662225 2555 scope.go:117] "RemoveContainer" containerID="f8692b6822a42ac57622d7f993cc0fe6293ade303beda66bb8ed2dacf82d713f" Sep 08 18:37:40.663369 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:40.663343 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 18:37:40.663446 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:40.663433 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 18:37:40.663483 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:40.663469 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 18:37:41.474007 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:41.473970 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:41.669463 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:41.669433 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" event={"ID":"ed6265b9-7057-49d9-85e6-9b08b0809029","Type":"ContainerStarted","Data":"2d8855908839af3b4d98f8070c9d8e487b78f8c978b5c177cd63b05baa84db02"} Sep 08 18:37:42.018710 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:42.017648 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:42.036619 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:42.036592 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-abd9a747-54b9ffd4d5-m9gsq"] Sep 08 18:37:42.036881 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:42.036854 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-abd9a747-54b9ffd4d5-m9gsq" podUID="e39243cb-6eb0-4718-b3a4-3e355f4c0a9c" containerName="payload-processing" containerID="cri-o://f37ee399a375c1f7954b4dd6ece9bb2c674717a36d183b35196c82e8478873f4" gracePeriod=30 Sep 08 18:37:42.102423 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:42.102386 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:42.194713 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:42.194081 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:42.212708 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:42.209738 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-abd9a747-rnwkm"] Sep 08 18:37:42.217108 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:42.217075 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:37:42.217273 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:42.217181 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-abd9a747-rnwkm"] Sep 08 18:37:42.217273 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:42.217180 2555 patch_prober.go:28] interesting pod/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Sep 08 18:37:42.217273 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:42.217235 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" podUID="ed6265b9-7057-49d9-85e6-9b08b0809029" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Sep 08 18:37:42.266665 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:42.266638 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:42.329450 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:42.329403 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97bc5cea-58f0-4a7e-9b43-06ac862fc49e" path="/var/lib/kubelet/pods/97bc5cea-58f0-4a7e-9b43-06ac862fc49e/volumes" Sep 08 18:37:42.335329 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:42.335298 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-abd9a747-54b9ffd4d5-m9gsq" Sep 08 18:37:42.360497 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:42.360465 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:42.424051 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:42.424020 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q72rb\" (UniqueName: \"kubernetes.io/projected/e39243cb-6eb0-4718-b3a4-3e355f4c0a9c-kube-api-access-q72rb\") pod \"e39243cb-6eb0-4718-b3a4-3e355f4c0a9c\" (UID: \"e39243cb-6eb0-4718-b3a4-3e355f4c0a9c\") " Sep 08 18:37:42.424212 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:42.424139 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e39243cb-6eb0-4718-b3a4-3e355f4c0a9c-plugins-config-volume\") pod \"e39243cb-6eb0-4718-b3a4-3e355f4c0a9c\" (UID: \"e39243cb-6eb0-4718-b3a4-3e355f4c0a9c\") " Sep 08 18:37:42.424496 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:42.424476 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e39243cb-6eb0-4718-b3a4-3e355f4c0a9c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e39243cb-6eb0-4718-b3a4-3e355f4c0a9c" (UID: "e39243cb-6eb0-4718-b3a4-3e355f4c0a9c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:37:42.426342 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:42.426318 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e39243cb-6eb0-4718-b3a4-3e355f4c0a9c-kube-api-access-q72rb" (OuterVolumeSpecName: "kube-api-access-q72rb") pod "e39243cb-6eb0-4718-b3a4-3e355f4c0a9c" (UID: "e39243cb-6eb0-4718-b3a4-3e355f4c0a9c"). InnerVolumeSpecName "kube-api-access-q72rb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:37:42.525232 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:42.525204 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e39243cb-6eb0-4718-b3a4-3e355f4c0a9c-plugins-config-volume\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:37:42.525232 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:42.525228 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q72rb\" (UniqueName: \"kubernetes.io/projected/e39243cb-6eb0-4718-b3a4-3e355f4c0a9c-kube-api-access-q72rb\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:37:42.676358 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:42.676319 2555 generic.go:356] "Generic (PLEG): container finished" podID="e39243cb-6eb0-4718-b3a4-3e355f4c0a9c" containerID="f37ee399a375c1f7954b4dd6ece9bb2c674717a36d183b35196c82e8478873f4" exitCode=0 Sep 08 18:37:42.676823 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:42.676367 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-abd9a747-54b9ffd4d5-m9gsq" Sep 08 18:37:42.676823 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:42.676371 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-abd9a747-54b9ffd4d5-m9gsq" event={"ID":"e39243cb-6eb0-4718-b3a4-3e355f4c0a9c","Type":"ContainerDied","Data":"f37ee399a375c1f7954b4dd6ece9bb2c674717a36d183b35196c82e8478873f4"} Sep 08 18:37:42.676823 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:42.676457 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-abd9a747-54b9ffd4d5-m9gsq" event={"ID":"e39243cb-6eb0-4718-b3a4-3e355f4c0a9c","Type":"ContainerDied","Data":"cff38206bf08c0126fd6f0c7716d9274ee1196accc132aa954114fd91915505f"} Sep 08 18:37:42.676823 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:42.676489 2555 scope.go:117] "RemoveContainer" containerID="f37ee399a375c1f7954b4dd6ece9bb2c674717a36d183b35196c82e8478873f4" Sep 08 18:37:42.686188 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:42.686173 2555 scope.go:117] "RemoveContainer" containerID="f37ee399a375c1f7954b4dd6ece9bb2c674717a36d183b35196c82e8478873f4" Sep 08 18:37:42.686462 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:37:42.686444 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f37ee399a375c1f7954b4dd6ece9bb2c674717a36d183b35196c82e8478873f4\": container with ID starting with f37ee399a375c1f7954b4dd6ece9bb2c674717a36d183b35196c82e8478873f4 not found: ID does not exist" containerID="f37ee399a375c1f7954b4dd6ece9bb2c674717a36d183b35196c82e8478873f4" Sep 08 18:37:42.686530 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:42.686471 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f37ee399a375c1f7954b4dd6ece9bb2c674717a36d183b35196c82e8478873f4"} err="failed to get container status \"f37ee399a375c1f7954b4dd6ece9bb2c674717a36d183b35196c82e8478873f4\": rpc error: code = NotFound desc = could not find container \"f37ee399a375c1f7954b4dd6ece9bb2c674717a36d183b35196c82e8478873f4\": container with ID starting with f37ee399a375c1f7954b4dd6ece9bb2c674717a36d183b35196c82e8478873f4 not found: ID does not exist" Sep 08 18:37:42.699436 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:42.699407 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-abd9a747-54b9ffd4d5-m9gsq"] Sep 08 18:37:42.703897 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:42.703874 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-abd9a747-54b9ffd4d5-m9gsq"] Sep 08 18:37:42.813101 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:42.813053 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:42.911797 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:42.911751 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:43.015558 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:43.015484 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:43.216927 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:43.216892 2555 patch_prober.go:28] interesting pod/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Sep 08 18:37:43.217087 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:43.216953 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" podUID="ed6265b9-7057-49d9-85e6-9b08b0809029" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Sep 08 18:37:43.330842 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:43.330814 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:43.549816 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:43.549779 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:44.216758 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:44.216723 2555 patch_prober.go:28] interesting pod/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Sep 08 18:37:44.217257 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:44.216777 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" podUID="ed6265b9-7057-49d9-85e6-9b08b0809029" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Sep 08 18:37:44.324254 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:44.324223 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e39243cb-6eb0-4718-b3a4-3e355f4c0a9c" path="/var/lib/kubelet/pods/e39243cb-6eb0-4718-b3a4-3e355f4c0a9c/volumes" Sep 08 18:37:45.217517 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:45.217477 2555 patch_prober.go:28] interesting pod/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Sep 08 18:37:45.217923 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:45.217540 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" podUID="ed6265b9-7057-49d9-85e6-9b08b0809029" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Sep 08 18:37:45.744827 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:45.744794 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:46.216893 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:46.216862 2555 patch_prober.go:28] interesting pod/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Sep 08 18:37:46.217069 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:46.216916 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" podUID="ed6265b9-7057-49d9-85e6-9b08b0809029" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Sep 08 18:37:46.276023 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:46.275989 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:46.372897 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:46.372866 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:46.473031 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:46.472955 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:46.573194 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:46.573160 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:46.901967 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:46.901938 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:47.035545 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:47.035517 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:47.141620 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:47.141591 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:47.221500 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:47.221420 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:37:47.222940 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:47.222914 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:37:47.224179 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:47.224140 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:37:47.224812 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:47.224788 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-ad9a714c-566fbbb86b-msc8m"] Sep 08 18:37:47.225790 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:47.225770 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97bc5cea-58f0-4a7e-9b43-06ac862fc49e" containerName="revoke-keys" Sep 08 18:37:47.225898 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:47.225878 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="97bc5cea-58f0-4a7e-9b43-06ac862fc49e" containerName="revoke-keys" Sep 08 18:37:47.226019 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:47.225909 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e39243cb-6eb0-4718-b3a4-3e355f4c0a9c" containerName="payload-processing" Sep 08 18:37:47.226019 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:47.225920 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="e39243cb-6eb0-4718-b3a4-3e355f4c0a9c" containerName="payload-processing" Sep 08 18:37:47.226120 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:47.226097 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="e39243cb-6eb0-4718-b3a4-3e355f4c0a9c" containerName="payload-processing" Sep 08 18:37:47.226120 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:47.226109 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="97bc5cea-58f0-4a7e-9b43-06ac862fc49e" containerName="revoke-keys" Sep 08 18:37:47.235023 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:47.235000 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-ad9a714c-566fbbb86b-msc8m" Sep 08 18:37:47.235522 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:47.235498 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-ad9a714c-566fbbb86b-msc8m"] Sep 08 18:37:47.238854 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:47.238832 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-ad9a714c\"" Sep 08 18:37:47.241675 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:47.241657 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-ad9a714c-dockercfg-d9nl6\"" Sep 08 18:37:47.266304 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:47.266280 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ae612b1-5865-4fa4-9185-cb1f37bbec17-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-ad9a714c-566fbbb86b-msc8m\" (UID: \"5ae612b1-5865-4fa4-9185-cb1f37bbec17\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-ad9a714c-566fbbb86b-msc8m" Sep 08 18:37:47.266422 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:47.266340 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vg8g\" (UniqueName: \"kubernetes.io/projected/5ae612b1-5865-4fa4-9185-cb1f37bbec17-kube-api-access-2vg8g\") pod \"payload-pre-processing-e2e-migrate-ad9a714c-566fbbb86b-msc8m\" (UID: \"5ae612b1-5865-4fa4-9185-cb1f37bbec17\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-ad9a714c-566fbbb86b-msc8m" Sep 08 18:37:47.308138 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:47.308107 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-ad9a714c-5ff44569c8-nx5lf"] Sep 08 18:37:47.316825 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:47.316801 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-ad9a714c-5ff44569c8-nx5lf" Sep 08 18:37:47.324735 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:47.324653 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-ad9a714c-5ff44569c8-nx5lf"] Sep 08 18:37:47.366902 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:47.366870 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vg8g\" (UniqueName: \"kubernetes.io/projected/5ae612b1-5865-4fa4-9185-cb1f37bbec17-kube-api-access-2vg8g\") pod \"payload-pre-processing-e2e-migrate-ad9a714c-566fbbb86b-msc8m\" (UID: \"5ae612b1-5865-4fa4-9185-cb1f37bbec17\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-ad9a714c-566fbbb86b-msc8m" Sep 08 18:37:47.367051 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:47.366950 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ae612b1-5865-4fa4-9185-cb1f37bbec17-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-ad9a714c-566fbbb86b-msc8m\" (UID: \"5ae612b1-5865-4fa4-9185-cb1f37bbec17\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-ad9a714c-566fbbb86b-msc8m" Sep 08 18:37:47.367051 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:47.367006 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrc5x\" (UniqueName: \"kubernetes.io/projected/efe5b51f-9f94-4cba-afb0-e215af95ba7d-kube-api-access-mrc5x\") pod \"payload-processing-e2e-migrate-ad9a714c-5ff44569c8-nx5lf\" (UID: \"efe5b51f-9f94-4cba-afb0-e215af95ba7d\") " pod="openshift-ingress/payload-processing-e2e-migrate-ad9a714c-5ff44569c8-nx5lf" Sep 08 18:37:47.367051 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:47.367039 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efe5b51f-9f94-4cba-afb0-e215af95ba7d-plugins-config-volume\") pod \"payload-processing-e2e-migrate-ad9a714c-5ff44569c8-nx5lf\" (UID: \"efe5b51f-9f94-4cba-afb0-e215af95ba7d\") " pod="openshift-ingress/payload-processing-e2e-migrate-ad9a714c-5ff44569c8-nx5lf" Sep 08 18:37:47.367625 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:47.367606 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ae612b1-5865-4fa4-9185-cb1f37bbec17-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-ad9a714c-566fbbb86b-msc8m\" (UID: \"5ae612b1-5865-4fa4-9185-cb1f37bbec17\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-ad9a714c-566fbbb86b-msc8m" Sep 08 18:37:47.374729 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:47.374679 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vg8g\" (UniqueName: \"kubernetes.io/projected/5ae612b1-5865-4fa4-9185-cb1f37bbec17-kube-api-access-2vg8g\") pod \"payload-pre-processing-e2e-migrate-ad9a714c-566fbbb86b-msc8m\" (UID: \"5ae612b1-5865-4fa4-9185-cb1f37bbec17\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-ad9a714c-566fbbb86b-msc8m" Sep 08 18:37:47.467987 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:47.467962 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrc5x\" (UniqueName: \"kubernetes.io/projected/efe5b51f-9f94-4cba-afb0-e215af95ba7d-kube-api-access-mrc5x\") pod \"payload-processing-e2e-migrate-ad9a714c-5ff44569c8-nx5lf\" (UID: \"efe5b51f-9f94-4cba-afb0-e215af95ba7d\") " pod="openshift-ingress/payload-processing-e2e-migrate-ad9a714c-5ff44569c8-nx5lf" Sep 08 18:37:47.468123 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:47.467996 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efe5b51f-9f94-4cba-afb0-e215af95ba7d-plugins-config-volume\") pod \"payload-processing-e2e-migrate-ad9a714c-5ff44569c8-nx5lf\" (UID: \"efe5b51f-9f94-4cba-afb0-e215af95ba7d\") " pod="openshift-ingress/payload-processing-e2e-migrate-ad9a714c-5ff44569c8-nx5lf" Sep 08 18:37:47.468592 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:47.468575 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efe5b51f-9f94-4cba-afb0-e215af95ba7d-plugins-config-volume\") pod \"payload-processing-e2e-migrate-ad9a714c-5ff44569c8-nx5lf\" (UID: \"efe5b51f-9f94-4cba-afb0-e215af95ba7d\") " pod="openshift-ingress/payload-processing-e2e-migrate-ad9a714c-5ff44569c8-nx5lf" Sep 08 18:37:47.477024 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:47.476973 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrc5x\" (UniqueName: \"kubernetes.io/projected/efe5b51f-9f94-4cba-afb0-e215af95ba7d-kube-api-access-mrc5x\") pod \"payload-processing-e2e-migrate-ad9a714c-5ff44569c8-nx5lf\" (UID: \"efe5b51f-9f94-4cba-afb0-e215af95ba7d\") " pod="openshift-ingress/payload-processing-e2e-migrate-ad9a714c-5ff44569c8-nx5lf" Sep 08 18:37:47.547831 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:47.547805 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-ad9a714c-566fbbb86b-msc8m" Sep 08 18:37:47.631337 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:47.631011 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-ad9a714c-5ff44569c8-nx5lf" Sep 08 18:37:47.660949 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:47.660915 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:47.824253 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:47.824227 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-ad9a714c-566fbbb86b-msc8m"] Sep 08 18:37:47.825118 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:37:47.825092 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ae612b1_5865_4fa4_9185_cb1f37bbec17.slice/crio-42e53647991a5f2d9d488182047b18643997486f6aa70812a67e2d886423c91e WatchSource:0}: Error finding container 42e53647991a5f2d9d488182047b18643997486f6aa70812a67e2d886423c91e: Status 404 returned error can't find the container with id 42e53647991a5f2d9d488182047b18643997486f6aa70812a67e2d886423c91e Sep 08 18:37:47.906532 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:47.906507 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-ad9a714c-5ff44569c8-nx5lf"] Sep 08 18:37:47.908232 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:37:47.908210 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefe5b51f_9f94_4cba_afb0_e215af95ba7d.slice/crio-991345a3cecd7222a1346cf53a91b005ac20b1da3c7e7bf0708b68b85c42c6b5 WatchSource:0}: Error finding container 991345a3cecd7222a1346cf53a91b005ac20b1da3c7e7bf0708b68b85c42c6b5: Status 404 returned error can't find the container with id 991345a3cecd7222a1346cf53a91b005ac20b1da3c7e7bf0708b68b85c42c6b5 Sep 08 18:37:48.364331 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:48.364299 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:48.460731 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:48.460702 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:37:48.722565 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:48.722475 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-ad9a714c-566fbbb86b-msc8m" event={"ID":"5ae612b1-5865-4fa4-9185-cb1f37bbec17","Type":"ContainerStarted","Data":"8ca397e6164deced3b72eb2058aa3a502a276782dbb1ab73b7630622b900225f"} Sep 08 18:37:48.722565 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:48.722533 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-ad9a714c-566fbbb86b-msc8m" event={"ID":"5ae612b1-5865-4fa4-9185-cb1f37bbec17","Type":"ContainerStarted","Data":"42e53647991a5f2d9d488182047b18643997486f6aa70812a67e2d886423c91e"} Sep 08 18:37:48.722842 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:48.722636 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-ad9a714c-566fbbb86b-msc8m" Sep 08 18:37:48.724187 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:48.724159 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-ad9a714c-5ff44569c8-nx5lf" event={"ID":"efe5b51f-9f94-4cba-afb0-e215af95ba7d","Type":"ContainerStarted","Data":"996986bd299f64e0a66539439d05a9d00137dee3931af2093d5875eeb6b967b8"} Sep 08 18:37:48.724316 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:48.724204 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-ad9a714c-5ff44569c8-nx5lf" event={"ID":"efe5b51f-9f94-4cba-afb0-e215af95ba7d","Type":"ContainerStarted","Data":"991345a3cecd7222a1346cf53a91b005ac20b1da3c7e7bf0708b68b85c42c6b5"} Sep 08 18:37:48.724382 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:48.724370 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-ad9a714c-5ff44569c8-nx5lf" Sep 08 18:37:48.737308 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:48.737260 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-ad9a714c-566fbbb86b-msc8m" podStartSLOduration=1.7372435579999999 podStartE2EDuration="1.737243558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:37:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:37:48.736969165 +0000 UTC m=+850.942954562" watchObservedRunningTime="2026-09-08 18:37:48.737243558 +0000 UTC m=+850.943228955" Sep 08 18:37:48.751960 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:48.751918 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-ad9a714c-5ff44569c8-nx5lf" podStartSLOduration=1.751858852 podStartE2EDuration="1.751858852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:37:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:37:48.751496846 +0000 UTC m=+850.957482244" watchObservedRunningTime="2026-09-08 18:37:48.751858852 +0000 UTC m=+850.957844249" Sep 08 18:37:56.530822 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:56.530784 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:37:59.732880 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:59.732840 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-ad9a714c-566fbbb86b-msc8m" Sep 08 18:37:59.733316 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:37:59.732995 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-ad9a714c-5ff44569c8-nx5lf" Sep 08 18:38:11.024537 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:11.024503 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:38:17.705259 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:17.705214 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:38:17.749900 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:17.749853 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-ad9a714c-5ff44569c8-nx5lf"] Sep 08 18:38:17.750176 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:17.750146 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-ad9a714c-5ff44569c8-nx5lf" podUID="efe5b51f-9f94-4cba-afb0-e215af95ba7d" containerName="payload-processing" containerID="cri-o://996986bd299f64e0a66539439d05a9d00137dee3931af2093d5875eeb6b967b8" gracePeriod=30 Sep 08 18:38:17.753976 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:17.753161 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-ad9a714c-566fbbb86b-msc8m"] Sep 08 18:38:17.753976 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:17.753378 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-ad9a714c-566fbbb86b-msc8m" podUID="5ae612b1-5865-4fa4-9185-cb1f37bbec17" containerName="payload-pre-processing" containerID="cri-o://8ca397e6164deced3b72eb2058aa3a502a276782dbb1ab73b7630622b900225f" gracePeriod=30 Sep 08 18:38:17.769655 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:17.769629 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:38:17.872315 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:17.872292 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:38:17.902002 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:17.901965 2555 generic.go:356] "Generic (PLEG): container finished" podID="5ae612b1-5865-4fa4-9185-cb1f37bbec17" containerID="8ca397e6164deced3b72eb2058aa3a502a276782dbb1ab73b7630622b900225f" exitCode=0 Sep 08 18:38:17.902198 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:17.902085 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-ad9a714c-566fbbb86b-msc8m" event={"ID":"5ae612b1-5865-4fa4-9185-cb1f37bbec17","Type":"ContainerDied","Data":"8ca397e6164deced3b72eb2058aa3a502a276782dbb1ab73b7630622b900225f"} Sep 08 18:38:17.904314 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:17.904292 2555 generic.go:356] "Generic (PLEG): container finished" podID="efe5b51f-9f94-4cba-afb0-e215af95ba7d" containerID="996986bd299f64e0a66539439d05a9d00137dee3931af2093d5875eeb6b967b8" exitCode=0 Sep 08 18:38:17.904415 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:17.904361 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-ad9a714c-5ff44569c8-nx5lf" event={"ID":"efe5b51f-9f94-4cba-afb0-e215af95ba7d","Type":"ContainerDied","Data":"996986bd299f64e0a66539439d05a9d00137dee3931af2093d5875eeb6b967b8"} Sep 08 18:38:17.939917 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:17.939881 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:38:18.020283 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.020247 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:38:18.037868 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.037849 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-ad9a714c-566fbbb86b-msc8m" Sep 08 18:38:18.050392 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.050368 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vg8g\" (UniqueName: \"kubernetes.io/projected/5ae612b1-5865-4fa4-9185-cb1f37bbec17-kube-api-access-2vg8g\") pod \"5ae612b1-5865-4fa4-9185-cb1f37bbec17\" (UID: \"5ae612b1-5865-4fa4-9185-cb1f37bbec17\") " Sep 08 18:38:18.050511 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.050429 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ae612b1-5865-4fa4-9185-cb1f37bbec17-plugins-config-volume\") pod \"5ae612b1-5865-4fa4-9185-cb1f37bbec17\" (UID: \"5ae612b1-5865-4fa4-9185-cb1f37bbec17\") " Sep 08 18:38:18.050795 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.050773 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ae612b1-5865-4fa4-9185-cb1f37bbec17-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5ae612b1-5865-4fa4-9185-cb1f37bbec17" (UID: "5ae612b1-5865-4fa4-9185-cb1f37bbec17"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:38:18.051199 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.051184 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-ad9a714c-5ff44569c8-nx5lf" Sep 08 18:38:18.052342 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.052321 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ae612b1-5865-4fa4-9185-cb1f37bbec17-kube-api-access-2vg8g" (OuterVolumeSpecName: "kube-api-access-2vg8g") pod "5ae612b1-5865-4fa4-9185-cb1f37bbec17" (UID: "5ae612b1-5865-4fa4-9185-cb1f37bbec17"). InnerVolumeSpecName "kube-api-access-2vg8g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:38:18.151079 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.151045 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efe5b51f-9f94-4cba-afb0-e215af95ba7d-plugins-config-volume\") pod \"efe5b51f-9f94-4cba-afb0-e215af95ba7d\" (UID: \"efe5b51f-9f94-4cba-afb0-e215af95ba7d\") " Sep 08 18:38:18.151265 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.151251 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vg8g\" (UniqueName: \"kubernetes.io/projected/5ae612b1-5865-4fa4-9185-cb1f37bbec17-kube-api-access-2vg8g\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:38:18.151308 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.151267 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ae612b1-5865-4fa4-9185-cb1f37bbec17-plugins-config-volume\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:38:18.151400 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.151378 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efe5b51f-9f94-4cba-afb0-e215af95ba7d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "efe5b51f-9f94-4cba-afb0-e215af95ba7d" (UID: "efe5b51f-9f94-4cba-afb0-e215af95ba7d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:38:18.252311 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.252217 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mrc5x\" (UniqueName: \"kubernetes.io/projected/efe5b51f-9f94-4cba-afb0-e215af95ba7d-kube-api-access-mrc5x\") pod \"efe5b51f-9f94-4cba-afb0-e215af95ba7d\" (UID: \"efe5b51f-9f94-4cba-afb0-e215af95ba7d\") " Sep 08 18:38:18.252464 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.252405 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efe5b51f-9f94-4cba-afb0-e215af95ba7d-plugins-config-volume\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:38:18.254408 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.254383 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efe5b51f-9f94-4cba-afb0-e215af95ba7d-kube-api-access-mrc5x" (OuterVolumeSpecName: "kube-api-access-mrc5x") pod "efe5b51f-9f94-4cba-afb0-e215af95ba7d" (UID: "efe5b51f-9f94-4cba-afb0-e215af95ba7d"). InnerVolumeSpecName "kube-api-access-mrc5x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:38:18.352917 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.352890 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrc5x\" (UniqueName: \"kubernetes.io/projected/efe5b51f-9f94-4cba-afb0-e215af95ba7d-kube-api-access-mrc5x\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:38:18.446332 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.446296 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:38:18.721028 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.720991 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp"] Sep 08 18:38:18.721599 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.721580 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ae612b1-5865-4fa4-9185-cb1f37bbec17" containerName="payload-pre-processing" Sep 08 18:38:18.721720 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.721601 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ae612b1-5865-4fa4-9185-cb1f37bbec17" containerName="payload-pre-processing" Sep 08 18:38:18.721720 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.721615 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efe5b51f-9f94-4cba-afb0-e215af95ba7d" containerName="payload-processing" Sep 08 18:38:18.721720 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.721622 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="efe5b51f-9f94-4cba-afb0-e215af95ba7d" containerName="payload-processing" Sep 08 18:38:18.721878 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.721779 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="efe5b51f-9f94-4cba-afb0-e215af95ba7d" containerName="payload-processing" Sep 08 18:38:18.721878 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.721801 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ae612b1-5865-4fa4-9185-cb1f37bbec17" containerName="payload-pre-processing" Sep 08 18:38:18.725176 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.725155 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:18.727072 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.727056 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-12c5e22e-openshift-default-dockercfg-mh845\"" Sep 08 18:38:18.738855 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.738832 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp"] Sep 08 18:38:18.748721 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.748669 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:38:18.757052 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.757030 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6aeaf180-2f92-446b-bd82-8325e7e7289b-istio-podinfo\") pod \"e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp\" (UID: \"6aeaf180-2f92-446b-bd82-8325e7e7289b\") " pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:18.757157 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.757059 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6aeaf180-2f92-446b-bd82-8325e7e7289b-istio-token\") pod \"e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp\" (UID: \"6aeaf180-2f92-446b-bd82-8325e7e7289b\") " pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:18.757157 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.757079 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6aeaf180-2f92-446b-bd82-8325e7e7289b-istiod-ca-cert\") pod \"e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp\" (UID: \"6aeaf180-2f92-446b-bd82-8325e7e7289b\") " pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:18.757157 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.757117 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6aeaf180-2f92-446b-bd82-8325e7e7289b-istio-envoy\") pod \"e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp\" (UID: \"6aeaf180-2f92-446b-bd82-8325e7e7289b\") " pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:18.757331 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.757206 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aeaf180-2f92-446b-bd82-8325e7e7289b-credential-socket\") pod \"e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp\" (UID: \"6aeaf180-2f92-446b-bd82-8325e7e7289b\") " pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:18.757331 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.757244 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7czh\" (UniqueName: \"kubernetes.io/projected/6aeaf180-2f92-446b-bd82-8325e7e7289b-kube-api-access-d7czh\") pod \"e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp\" (UID: \"6aeaf180-2f92-446b-bd82-8325e7e7289b\") " pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:18.757331 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.757307 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aeaf180-2f92-446b-bd82-8325e7e7289b-workload-socket\") pod \"e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp\" (UID: \"6aeaf180-2f92-446b-bd82-8325e7e7289b\") " pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:18.757447 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.757341 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6aeaf180-2f92-446b-bd82-8325e7e7289b-workload-certs\") pod \"e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp\" (UID: \"6aeaf180-2f92-446b-bd82-8325e7e7289b\") " pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:18.757447 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.757386 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6aeaf180-2f92-446b-bd82-8325e7e7289b-istio-data\") pod \"e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp\" (UID: \"6aeaf180-2f92-446b-bd82-8325e7e7289b\") " pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:18.852547 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.852518 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:38:18.857949 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.857925 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6aeaf180-2f92-446b-bd82-8325e7e7289b-istio-podinfo\") pod \"e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp\" (UID: \"6aeaf180-2f92-446b-bd82-8325e7e7289b\") " pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:18.858028 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.857955 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6aeaf180-2f92-446b-bd82-8325e7e7289b-istio-token\") pod \"e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp\" (UID: \"6aeaf180-2f92-446b-bd82-8325e7e7289b\") " pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:18.858028 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.857973 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6aeaf180-2f92-446b-bd82-8325e7e7289b-istiod-ca-cert\") pod \"e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp\" (UID: \"6aeaf180-2f92-446b-bd82-8325e7e7289b\") " pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:18.858028 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.857994 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6aeaf180-2f92-446b-bd82-8325e7e7289b-istio-envoy\") pod \"e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp\" (UID: \"6aeaf180-2f92-446b-bd82-8325e7e7289b\") " pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:18.858028 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.858016 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aeaf180-2f92-446b-bd82-8325e7e7289b-credential-socket\") pod \"e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp\" (UID: \"6aeaf180-2f92-446b-bd82-8325e7e7289b\") " pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:18.858280 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.858166 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7czh\" (UniqueName: \"kubernetes.io/projected/6aeaf180-2f92-446b-bd82-8325e7e7289b-kube-api-access-d7czh\") pod \"e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp\" (UID: \"6aeaf180-2f92-446b-bd82-8325e7e7289b\") " pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:18.858280 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.858222 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aeaf180-2f92-446b-bd82-8325e7e7289b-workload-socket\") pod \"e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp\" (UID: \"6aeaf180-2f92-446b-bd82-8325e7e7289b\") " pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:18.858280 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.858261 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6aeaf180-2f92-446b-bd82-8325e7e7289b-workload-certs\") pod \"e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp\" (UID: \"6aeaf180-2f92-446b-bd82-8325e7e7289b\") " pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:18.858440 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.858307 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6aeaf180-2f92-446b-bd82-8325e7e7289b-istio-data\") pod \"e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp\" (UID: \"6aeaf180-2f92-446b-bd82-8325e7e7289b\") " pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:18.858440 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.858372 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aeaf180-2f92-446b-bd82-8325e7e7289b-credential-socket\") pod \"e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp\" (UID: \"6aeaf180-2f92-446b-bd82-8325e7e7289b\") " pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:18.858568 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.858541 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aeaf180-2f92-446b-bd82-8325e7e7289b-workload-socket\") pod \"e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp\" (UID: \"6aeaf180-2f92-446b-bd82-8325e7e7289b\") " pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:18.858627 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.858615 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6aeaf180-2f92-446b-bd82-8325e7e7289b-workload-certs\") pod \"e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp\" (UID: \"6aeaf180-2f92-446b-bd82-8325e7e7289b\") " pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:18.858681 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.858660 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6aeaf180-2f92-446b-bd82-8325e7e7289b-istio-data\") pod \"e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp\" (UID: \"6aeaf180-2f92-446b-bd82-8325e7e7289b\") " pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:18.858776 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.858757 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6aeaf180-2f92-446b-bd82-8325e7e7289b-istiod-ca-cert\") pod \"e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp\" (UID: \"6aeaf180-2f92-446b-bd82-8325e7e7289b\") " pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:18.860368 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.860348 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6aeaf180-2f92-446b-bd82-8325e7e7289b-istio-envoy\") pod \"e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp\" (UID: \"6aeaf180-2f92-446b-bd82-8325e7e7289b\") " pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:18.860572 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.860555 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6aeaf180-2f92-446b-bd82-8325e7e7289b-istio-podinfo\") pod \"e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp\" (UID: \"6aeaf180-2f92-446b-bd82-8325e7e7289b\") " pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:18.865678 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.865658 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6aeaf180-2f92-446b-bd82-8325e7e7289b-istio-token\") pod \"e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp\" (UID: \"6aeaf180-2f92-446b-bd82-8325e7e7289b\") " pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:18.865902 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.865876 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7czh\" (UniqueName: \"kubernetes.io/projected/6aeaf180-2f92-446b-bd82-8325e7e7289b-kube-api-access-d7czh\") pod \"e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp\" (UID: \"6aeaf180-2f92-446b-bd82-8325e7e7289b\") " pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:18.912031 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.912002 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-ad9a714c-566fbbb86b-msc8m" Sep 08 18:38:18.912215 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.912004 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-ad9a714c-566fbbb86b-msc8m" event={"ID":"5ae612b1-5865-4fa4-9185-cb1f37bbec17","Type":"ContainerDied","Data":"42e53647991a5f2d9d488182047b18643997486f6aa70812a67e2d886423c91e"} Sep 08 18:38:18.912215 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.912142 2555 scope.go:117] "RemoveContainer" containerID="8ca397e6164deced3b72eb2058aa3a502a276782dbb1ab73b7630622b900225f" Sep 08 18:38:18.913615 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.913593 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-ad9a714c-5ff44569c8-nx5lf" event={"ID":"efe5b51f-9f94-4cba-afb0-e215af95ba7d","Type":"ContainerDied","Data":"991345a3cecd7222a1346cf53a91b005ac20b1da3c7e7bf0708b68b85c42c6b5"} Sep 08 18:38:18.913717 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.913606 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-ad9a714c-5ff44569c8-nx5lf" Sep 08 18:38:18.921618 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.921597 2555 scope.go:117] "RemoveContainer" containerID="996986bd299f64e0a66539439d05a9d00137dee3931af2093d5875eeb6b967b8" Sep 08 18:38:18.934613 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.934591 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-ad9a714c-566fbbb86b-msc8m"] Sep 08 18:38:18.938860 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.938839 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-ad9a714c-566fbbb86b-msc8m"] Sep 08 18:38:18.948406 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.948387 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-ad9a714c-5ff44569c8-nx5lf"] Sep 08 18:38:18.954822 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:18.954800 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-ad9a714c-5ff44569c8-nx5lf"] Sep 08 18:38:19.034880 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:19.034785 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:19.317925 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:19.317900 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp"] Sep 08 18:38:19.319611 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:38:19.319577 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6aeaf180_2f92_446b_bd82_8325e7e7289b.slice/crio-aaddec873a678da82c909957f8b77f9fc24cd442e81d74f3f8d80fa853f6dc37 WatchSource:0}: Error finding container aaddec873a678da82c909957f8b77f9fc24cd442e81d74f3f8d80fa853f6dc37: Status 404 returned error can't find the container with id aaddec873a678da82c909957f8b77f9fc24cd442e81d74f3f8d80fa853f6dc37 Sep 08 18:38:19.321720 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:19.321668 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 18:38:19.321818 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:19.321752 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 18:38:19.321818 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:19.321780 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 18:38:19.922613 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:19.922578 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" event={"ID":"6aeaf180-2f92-446b-bd82-8325e7e7289b","Type":"ContainerStarted","Data":"b9c578ea16a3cadbb24ea04c7a684172eb95b44e708ac00cafb608787cff86cb"} Sep 08 18:38:19.923092 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:19.922634 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" event={"ID":"6aeaf180-2f92-446b-bd82-8325e7e7289b","Type":"ContainerStarted","Data":"aaddec873a678da82c909957f8b77f9fc24cd442e81d74f3f8d80fa853f6dc37"} Sep 08 18:38:19.944858 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:19.944809 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" podStartSLOduration=1.94479393 podStartE2EDuration="1.94479393s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:38:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:38:19.942077741 +0000 UTC m=+882.148063138" watchObservedRunningTime="2026-09-08 18:38:19.94479393 +0000 UTC m=+882.150779384" Sep 08 18:38:20.035562 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:20.035528 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:20.040931 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:20.040906 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:20.324303 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:20.324278 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ae612b1-5865-4fa4-9185-cb1f37bbec17" path="/var/lib/kubelet/pods/5ae612b1-5865-4fa4-9185-cb1f37bbec17/volumes" Sep 08 18:38:20.324624 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:20.324612 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efe5b51f-9f94-4cba-afb0-e215af95ba7d" path="/var/lib/kubelet/pods/efe5b51f-9f94-4cba-afb0-e215af95ba7d/volumes" Sep 08 18:38:20.930360 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:20.930329 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:20.931212 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:20.931183 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:21.259092 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:21.259013 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:38:21.359900 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:21.359865 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:38:21.522491 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:21.522415 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:38:22.482129 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:22.482098 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:38:22.585382 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:22.585350 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:38:22.683164 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:22.683136 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:38:22.970421 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:22.970388 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:38:23.069959 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:23.069927 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:38:23.915012 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:23.914984 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:38:26.127066 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:26.127025 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:38:35.977660 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:35.976259 2555 patch_prober.go:28] interesting pod/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": read tcp 10.132.0.2:34084->10.132.0.56:15021: read: connection reset by peer" start-of-body= Sep 08 18:38:35.977660 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:35.976334 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" podUID="6aeaf180-2f92-446b-bd82-8325e7e7289b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": read tcp 10.132.0.2:34084->10.132.0.56:15021: read: connection reset by peer" Sep 08 18:38:36.025470 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:36.025445 2555 generic.go:356] "Generic (PLEG): container finished" podID="6aeaf180-2f92-446b-bd82-8325e7e7289b" containerID="b9c578ea16a3cadbb24ea04c7a684172eb95b44e708ac00cafb608787cff86cb" exitCode=0 Sep 08 18:38:36.025579 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:36.025524 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" event={"ID":"6aeaf180-2f92-446b-bd82-8325e7e7289b","Type":"ContainerDied","Data":"b9c578ea16a3cadbb24ea04c7a684172eb95b44e708ac00cafb608787cff86cb"} Sep 08 18:38:36.025945 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:36.025930 2555 scope.go:117] "RemoveContainer" containerID="b9c578ea16a3cadbb24ea04c7a684172eb95b44e708ac00cafb608787cff86cb" Sep 08 18:38:36.027184 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:36.027154 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 18:38:36.027288 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:36.027268 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 18:38:36.027357 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:36.027324 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 18:38:37.033259 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:37.033228 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" event={"ID":"6aeaf180-2f92-446b-bd82-8325e7e7289b","Type":"ContainerStarted","Data":"39b7415ef12506efef3688c212b7377c25893cb3b33e519eef4ed62c4be3385b"} Sep 08 18:38:37.035717 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:37.035703 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:37.035929 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:37.035908 2555 patch_prober.go:28] interesting pod/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Sep 08 18:38:37.035974 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:37.035957 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" podUID="6aeaf180-2f92-446b-bd82-8325e7e7289b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Sep 08 18:38:38.035873 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:38.035829 2555 patch_prober.go:28] interesting pod/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Sep 08 18:38:38.036333 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:38.035900 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" podUID="6aeaf180-2f92-446b-bd82-8325e7e7289b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Sep 08 18:38:38.338456 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:38.338426 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9n28f_911c61b8-766b-4ae8-b046-784e22e85889/console-operator/1.log" Sep 08 18:38:38.338631 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:38.338577 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9n28f_911c61b8-766b-4ae8-b046-784e22e85889/console-operator/1.log" Sep 08 18:38:39.035256 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:39.035220 2555 patch_prober.go:28] interesting pod/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Sep 08 18:38:39.035433 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:39.035278 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" podUID="6aeaf180-2f92-446b-bd82-8325e7e7289b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Sep 08 18:38:40.035913 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:40.035879 2555 patch_prober.go:28] interesting pod/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Sep 08 18:38:40.036289 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:40.035931 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" podUID="6aeaf180-2f92-446b-bd82-8325e7e7289b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Sep 08 18:38:41.035460 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:41.035421 2555 patch_prober.go:28] interesting pod/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Sep 08 18:38:41.035641 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:41.035477 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" podUID="6aeaf180-2f92-446b-bd82-8325e7e7289b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Sep 08 18:38:42.038850 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:42.038811 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:42.039296 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:42.039199 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:42.039850 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:42.039831 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:42.555773 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:42.555738 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:38:46.433517 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:46.433475 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:38:52.948120 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:52.948089 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:38:53.014340 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:53.014301 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:38:53.114986 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:53.114959 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:38:53.171218 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:53.171192 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:38:54.266111 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:54.266075 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp"] Sep 08 18:38:54.266475 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:54.266280 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" podUID="6aeaf180-2f92-446b-bd82-8325e7e7289b" containerName="istio-proxy" containerID="cri-o://39b7415ef12506efef3688c212b7377c25893cb3b33e519eef4ed62c4be3385b" gracePeriod=30 Sep 08 18:38:54.292968 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:54.292943 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:38:54.688033 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:54.688007 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:38:54.997563 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:54.997469 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:38:55.323098 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:55.323062 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:38:57.041601 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:57.041562 2555 patch_prober.go:28] interesting pod/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 08 18:38:57.041979 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:57.041630 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" podUID="6aeaf180-2f92-446b-bd82-8325e7e7289b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 08 18:38:57.604088 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:57.604027 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:38:57.694987 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:57.694955 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:38:58.856926 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:58.856893 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:38:58.957987 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:58.957955 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:38:59.014944 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.014919 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:38:59.419852 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.419813 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:38:59.509929 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.509902 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-18913a0c-df86496c8-vktz6"] Sep 08 18:38:59.511476 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.511450 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:38:59.513813 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.513794 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-18913a0c-df86496c8-vktz6" Sep 08 18:38:59.515847 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.515831 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-18913a0c\"" Sep 08 18:38:59.522962 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.522941 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-18913a0c-df86496c8-vktz6"] Sep 08 18:38:59.537232 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.537173 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:38:59.559001 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.558981 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-18913a0c-6d84f859df-9bj4k"] Sep 08 18:38:59.559348 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.559336 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6aeaf180-2f92-446b-bd82-8325e7e7289b" containerName="istio-proxy" Sep 08 18:38:59.559390 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.559349 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="6aeaf180-2f92-446b-bd82-8325e7e7289b" containerName="istio-proxy" Sep 08 18:38:59.559439 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.559430 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="6aeaf180-2f92-446b-bd82-8325e7e7289b" containerName="istio-proxy" Sep 08 18:38:59.559474 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.559441 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="6aeaf180-2f92-446b-bd82-8325e7e7289b" containerName="istio-proxy" Sep 08 18:38:59.562340 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.562327 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-18913a0c-6d84f859df-9bj4k" Sep 08 18:38:59.564508 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.564493 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-18913a0c-dockercfg-dx8np\"" Sep 08 18:38:59.584528 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.584504 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-18913a0c-6d84f859df-9bj4k"] Sep 08 18:38:59.644944 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.644918 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aeaf180-2f92-446b-bd82-8325e7e7289b-workload-socket\") pod \"6aeaf180-2f92-446b-bd82-8325e7e7289b\" (UID: \"6aeaf180-2f92-446b-bd82-8325e7e7289b\") " Sep 08 18:38:59.645098 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.644955 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aeaf180-2f92-446b-bd82-8325e7e7289b-credential-socket\") pod \"6aeaf180-2f92-446b-bd82-8325e7e7289b\" (UID: \"6aeaf180-2f92-446b-bd82-8325e7e7289b\") " Sep 08 18:38:59.645098 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.644990 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6aeaf180-2f92-446b-bd82-8325e7e7289b-istio-envoy\") pod \"6aeaf180-2f92-446b-bd82-8325e7e7289b\" (UID: \"6aeaf180-2f92-446b-bd82-8325e7e7289b\") " Sep 08 18:38:59.645098 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.645027 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6aeaf180-2f92-446b-bd82-8325e7e7289b-istio-token\") pod \"6aeaf180-2f92-446b-bd82-8325e7e7289b\" (UID: \"6aeaf180-2f92-446b-bd82-8325e7e7289b\") " Sep 08 18:38:59.645098 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.645051 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d7czh\" (UniqueName: \"kubernetes.io/projected/6aeaf180-2f92-446b-bd82-8325e7e7289b-kube-api-access-d7czh\") pod \"6aeaf180-2f92-446b-bd82-8325e7e7289b\" (UID: \"6aeaf180-2f92-446b-bd82-8325e7e7289b\") " Sep 08 18:38:59.645098 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.645093 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6aeaf180-2f92-446b-bd82-8325e7e7289b-istio-podinfo\") pod \"6aeaf180-2f92-446b-bd82-8325e7e7289b\" (UID: \"6aeaf180-2f92-446b-bd82-8325e7e7289b\") " Sep 08 18:38:59.645356 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.645143 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6aeaf180-2f92-446b-bd82-8325e7e7289b-workload-certs\") pod \"6aeaf180-2f92-446b-bd82-8325e7e7289b\" (UID: \"6aeaf180-2f92-446b-bd82-8325e7e7289b\") " Sep 08 18:38:59.645356 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.645199 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6aeaf180-2f92-446b-bd82-8325e7e7289b-istiod-ca-cert\") pod \"6aeaf180-2f92-446b-bd82-8325e7e7289b\" (UID: \"6aeaf180-2f92-446b-bd82-8325e7e7289b\") " Sep 08 18:38:59.645356 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.645227 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6aeaf180-2f92-446b-bd82-8325e7e7289b-istio-data\") pod \"6aeaf180-2f92-446b-bd82-8325e7e7289b\" (UID: \"6aeaf180-2f92-446b-bd82-8325e7e7289b\") " Sep 08 18:38:59.645356 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.645242 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aeaf180-2f92-446b-bd82-8325e7e7289b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6aeaf180-2f92-446b-bd82-8325e7e7289b" (UID: "6aeaf180-2f92-446b-bd82-8325e7e7289b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:38:59.645356 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.645286 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aeaf180-2f92-446b-bd82-8325e7e7289b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6aeaf180-2f92-446b-bd82-8325e7e7289b" (UID: "6aeaf180-2f92-446b-bd82-8325e7e7289b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:38:59.645614 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.645388 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aeaf180-2f92-446b-bd82-8325e7e7289b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6aeaf180-2f92-446b-bd82-8325e7e7289b" (UID: "6aeaf180-2f92-446b-bd82-8325e7e7289b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:38:59.645614 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.645482 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbdj6\" (UniqueName: \"kubernetes.io/projected/cbd304f4-2c6c-4ddb-8f0d-ada5ca55b96a-kube-api-access-bbdj6\") pod \"payload-pre-processing-e2e-derive-18913a0c-df86496c8-vktz6\" (UID: \"cbd304f4-2c6c-4ddb-8f0d-ada5ca55b96a\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-18913a0c-df86496c8-vktz6" Sep 08 18:38:59.645614 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.645593 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6aeaf180-2f92-446b-bd82-8325e7e7289b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6aeaf180-2f92-446b-bd82-8325e7e7289b" (UID: "6aeaf180-2f92-446b-bd82-8325e7e7289b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:38:59.645614 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.645594 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aeaf180-2f92-446b-bd82-8325e7e7289b-istio-data" (OuterVolumeSpecName: "istio-data") pod "6aeaf180-2f92-446b-bd82-8325e7e7289b" (UID: "6aeaf180-2f92-446b-bd82-8325e7e7289b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:38:59.645809 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.645656 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cbd304f4-2c6c-4ddb-8f0d-ada5ca55b96a-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-18913a0c-df86496c8-vktz6\" (UID: \"cbd304f4-2c6c-4ddb-8f0d-ada5ca55b96a\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-18913a0c-df86496c8-vktz6" Sep 08 18:38:59.645809 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.645795 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6aeaf180-2f92-446b-bd82-8325e7e7289b-workload-certs\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:38:59.645875 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.645816 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6aeaf180-2f92-446b-bd82-8325e7e7289b-istiod-ca-cert\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:38:59.645875 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.645828 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6aeaf180-2f92-446b-bd82-8325e7e7289b-istio-data\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:38:59.645875 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.645841 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aeaf180-2f92-446b-bd82-8325e7e7289b-workload-socket\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:38:59.645875 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.645855 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aeaf180-2f92-446b-bd82-8325e7e7289b-credential-socket\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:38:59.647443 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.647415 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6aeaf180-2f92-446b-bd82-8325e7e7289b-kube-api-access-d7czh" (OuterVolumeSpecName: "kube-api-access-d7czh") pod "6aeaf180-2f92-446b-bd82-8325e7e7289b" (UID: "6aeaf180-2f92-446b-bd82-8325e7e7289b"). InnerVolumeSpecName "kube-api-access-d7czh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:38:59.647561 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.647452 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aeaf180-2f92-446b-bd82-8325e7e7289b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6aeaf180-2f92-446b-bd82-8325e7e7289b" (UID: "6aeaf180-2f92-446b-bd82-8325e7e7289b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:38:59.647892 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.647874 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6aeaf180-2f92-446b-bd82-8325e7e7289b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6aeaf180-2f92-446b-bd82-8325e7e7289b" (UID: "6aeaf180-2f92-446b-bd82-8325e7e7289b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 08 18:38:59.647970 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.647915 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6aeaf180-2f92-446b-bd82-8325e7e7289b-istio-token" (OuterVolumeSpecName: "istio-token") pod "6aeaf180-2f92-446b-bd82-8325e7e7289b" (UID: "6aeaf180-2f92-446b-bd82-8325e7e7289b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:38:59.746638 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.746615 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cbd304f4-2c6c-4ddb-8f0d-ada5ca55b96a-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-18913a0c-df86496c8-vktz6\" (UID: \"cbd304f4-2c6c-4ddb-8f0d-ada5ca55b96a\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-18913a0c-df86496c8-vktz6" Sep 08 18:38:59.746750 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.746644 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3235575-cac1-402c-bc8f-c02a34352a38-plugins-config-volume\") pod \"payload-processing-e2e-derive-18913a0c-6d84f859df-9bj4k\" (UID: \"d3235575-cac1-402c-bc8f-c02a34352a38\") " pod="openshift-ingress/payload-processing-e2e-derive-18913a0c-6d84f859df-9bj4k" Sep 08 18:38:59.746750 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.746660 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5wxx\" (UniqueName: \"kubernetes.io/projected/d3235575-cac1-402c-bc8f-c02a34352a38-kube-api-access-l5wxx\") pod \"payload-processing-e2e-derive-18913a0c-6d84f859df-9bj4k\" (UID: \"d3235575-cac1-402c-bc8f-c02a34352a38\") " pod="openshift-ingress/payload-processing-e2e-derive-18913a0c-6d84f859df-9bj4k" Sep 08 18:38:59.746828 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.746757 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbdj6\" (UniqueName: \"kubernetes.io/projected/cbd304f4-2c6c-4ddb-8f0d-ada5ca55b96a-kube-api-access-bbdj6\") pod \"payload-pre-processing-e2e-derive-18913a0c-df86496c8-vktz6\" (UID: \"cbd304f4-2c6c-4ddb-8f0d-ada5ca55b96a\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-18913a0c-df86496c8-vktz6" Sep 08 18:38:59.746862 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.746829 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6aeaf180-2f92-446b-bd82-8325e7e7289b-istio-podinfo\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:38:59.746862 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.746840 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6aeaf180-2f92-446b-bd82-8325e7e7289b-istio-envoy\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:38:59.746862 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.746853 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6aeaf180-2f92-446b-bd82-8325e7e7289b-istio-token\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:38:59.746954 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.746865 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7czh\" (UniqueName: \"kubernetes.io/projected/6aeaf180-2f92-446b-bd82-8325e7e7289b-kube-api-access-d7czh\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:38:59.747241 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.747224 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cbd304f4-2c6c-4ddb-8f0d-ada5ca55b96a-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-18913a0c-df86496c8-vktz6\" (UID: \"cbd304f4-2c6c-4ddb-8f0d-ada5ca55b96a\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-18913a0c-df86496c8-vktz6" Sep 08 18:38:59.755654 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.755628 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbdj6\" (UniqueName: \"kubernetes.io/projected/cbd304f4-2c6c-4ddb-8f0d-ada5ca55b96a-kube-api-access-bbdj6\") pod \"payload-pre-processing-e2e-derive-18913a0c-df86496c8-vktz6\" (UID: \"cbd304f4-2c6c-4ddb-8f0d-ada5ca55b96a\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-18913a0c-df86496c8-vktz6" Sep 08 18:38:59.843294 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.843244 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-18913a0c-df86496c8-vktz6" Sep 08 18:38:59.847286 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.847260 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3235575-cac1-402c-bc8f-c02a34352a38-plugins-config-volume\") pod \"payload-processing-e2e-derive-18913a0c-6d84f859df-9bj4k\" (UID: \"d3235575-cac1-402c-bc8f-c02a34352a38\") " pod="openshift-ingress/payload-processing-e2e-derive-18913a0c-6d84f859df-9bj4k" Sep 08 18:38:59.847403 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.847292 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5wxx\" (UniqueName: \"kubernetes.io/projected/d3235575-cac1-402c-bc8f-c02a34352a38-kube-api-access-l5wxx\") pod \"payload-processing-e2e-derive-18913a0c-6d84f859df-9bj4k\" (UID: \"d3235575-cac1-402c-bc8f-c02a34352a38\") " pod="openshift-ingress/payload-processing-e2e-derive-18913a0c-6d84f859df-9bj4k" Sep 08 18:38:59.847877 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.847856 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3235575-cac1-402c-bc8f-c02a34352a38-plugins-config-volume\") pod \"payload-processing-e2e-derive-18913a0c-6d84f859df-9bj4k\" (UID: \"d3235575-cac1-402c-bc8f-c02a34352a38\") " pod="openshift-ingress/payload-processing-e2e-derive-18913a0c-6d84f859df-9bj4k" Sep 08 18:38:59.854609 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.854592 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5wxx\" (UniqueName: \"kubernetes.io/projected/d3235575-cac1-402c-bc8f-c02a34352a38-kube-api-access-l5wxx\") pod \"payload-processing-e2e-derive-18913a0c-6d84f859df-9bj4k\" (UID: \"d3235575-cac1-402c-bc8f-c02a34352a38\") " pod="openshift-ingress/payload-processing-e2e-derive-18913a0c-6d84f859df-9bj4k" Sep 08 18:38:59.874058 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:38:59.874035 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-18913a0c-6d84f859df-9bj4k" Sep 08 18:39:00.115477 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:00.115449 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-18913a0c-df86496c8-vktz6"] Sep 08 18:39:00.116665 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:39:00.116641 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbd304f4_2c6c_4ddb_8f0d_ada5ca55b96a.slice/crio-8bfa19db24db270c23eea5c4e6a9d4dfcc0dd542356c8e4bbe32af990accce63 WatchSource:0}: Error finding container 8bfa19db24db270c23eea5c4e6a9d4dfcc0dd542356c8e4bbe32af990accce63: Status 404 returned error can't find the container with id 8bfa19db24db270c23eea5c4e6a9d4dfcc0dd542356c8e4bbe32af990accce63 Sep 08 18:39:00.157938 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:00.157912 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-18913a0c-6d84f859df-9bj4k"] Sep 08 18:39:00.166112 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:00.166090 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-18913a0c-df86496c8-vktz6" event={"ID":"cbd304f4-2c6c-4ddb-8f0d-ada5ca55b96a","Type":"ContainerStarted","Data":"8bfa19db24db270c23eea5c4e6a9d4dfcc0dd542356c8e4bbe32af990accce63"} Sep 08 18:39:00.167751 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:39:00.167732 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3235575_cac1_402c_bc8f_c02a34352a38.slice/crio-84e8371cd3333e35b0ded879707ce1bf9ed8bfb706f07897e6a1624ae0e01191 WatchSource:0}: Error finding container 84e8371cd3333e35b0ded879707ce1bf9ed8bfb706f07897e6a1624ae0e01191: Status 404 returned error can't find the container with id 84e8371cd3333e35b0ded879707ce1bf9ed8bfb706f07897e6a1624ae0e01191 Sep 08 18:39:00.168029 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:00.168013 2555 generic.go:356] "Generic (PLEG): container finished" podID="6aeaf180-2f92-446b-bd82-8325e7e7289b" containerID="39b7415ef12506efef3688c212b7377c25893cb3b33e519eef4ed62c4be3385b" exitCode=0 Sep 08 18:39:00.168127 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:00.168097 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" Sep 08 18:39:00.168190 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:00.168097 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" event={"ID":"6aeaf180-2f92-446b-bd82-8325e7e7289b","Type":"ContainerDied","Data":"39b7415ef12506efef3688c212b7377c25893cb3b33e519eef4ed62c4be3385b"} Sep 08 18:39:00.168244 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:00.168217 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp" event={"ID":"6aeaf180-2f92-446b-bd82-8325e7e7289b","Type":"ContainerDied","Data":"aaddec873a678da82c909957f8b77f9fc24cd442e81d74f3f8d80fa853f6dc37"} Sep 08 18:39:00.168244 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:00.168238 2555 scope.go:117] "RemoveContainer" containerID="39b7415ef12506efef3688c212b7377c25893cb3b33e519eef4ed62c4be3385b" Sep 08 18:39:00.179795 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:00.179782 2555 scope.go:117] "RemoveContainer" containerID="b9c578ea16a3cadbb24ea04c7a684172eb95b44e708ac00cafb608787cff86cb" Sep 08 18:39:00.189288 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:00.189257 2555 scope.go:117] "RemoveContainer" containerID="39b7415ef12506efef3688c212b7377c25893cb3b33e519eef4ed62c4be3385b" Sep 08 18:39:00.189574 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:39:00.189556 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39b7415ef12506efef3688c212b7377c25893cb3b33e519eef4ed62c4be3385b\": container with ID starting with 39b7415ef12506efef3688c212b7377c25893cb3b33e519eef4ed62c4be3385b not found: ID does not exist" containerID="39b7415ef12506efef3688c212b7377c25893cb3b33e519eef4ed62c4be3385b" Sep 08 18:39:00.189625 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:00.189583 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39b7415ef12506efef3688c212b7377c25893cb3b33e519eef4ed62c4be3385b"} err="failed to get container status \"39b7415ef12506efef3688c212b7377c25893cb3b33e519eef4ed62c4be3385b\": rpc error: code = NotFound desc = could not find container \"39b7415ef12506efef3688c212b7377c25893cb3b33e519eef4ed62c4be3385b\": container with ID starting with 39b7415ef12506efef3688c212b7377c25893cb3b33e519eef4ed62c4be3385b not found: ID does not exist" Sep 08 18:39:00.189625 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:00.189601 2555 scope.go:117] "RemoveContainer" containerID="b9c578ea16a3cadbb24ea04c7a684172eb95b44e708ac00cafb608787cff86cb" Sep 08 18:39:00.189932 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:39:00.189911 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9c578ea16a3cadbb24ea04c7a684172eb95b44e708ac00cafb608787cff86cb\": container with ID starting with b9c578ea16a3cadbb24ea04c7a684172eb95b44e708ac00cafb608787cff86cb not found: ID does not exist" containerID="b9c578ea16a3cadbb24ea04c7a684172eb95b44e708ac00cafb608787cff86cb" Sep 08 18:39:00.190004 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:00.189944 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9c578ea16a3cadbb24ea04c7a684172eb95b44e708ac00cafb608787cff86cb"} err="failed to get container status \"b9c578ea16a3cadbb24ea04c7a684172eb95b44e708ac00cafb608787cff86cb\": rpc error: code = NotFound desc = could not find container \"b9c578ea16a3cadbb24ea04c7a684172eb95b44e708ac00cafb608787cff86cb\": container with ID starting with b9c578ea16a3cadbb24ea04c7a684172eb95b44e708ac00cafb608787cff86cb not found: ID does not exist" Sep 08 18:39:00.196554 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:00.196534 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp"] Sep 08 18:39:00.200062 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:00.200043 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-12c5e22e-openshift-default-84496bfcbb-hfcfp"] Sep 08 18:39:00.325081 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:00.325048 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6aeaf180-2f92-446b-bd82-8325e7e7289b" path="/var/lib/kubelet/pods/6aeaf180-2f92-446b-bd82-8325e7e7289b/volumes" Sep 08 18:39:01.176165 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:01.176135 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-18913a0c-6d84f859df-9bj4k" event={"ID":"d3235575-cac1-402c-bc8f-c02a34352a38","Type":"ContainerStarted","Data":"4c350d2874b051eb363fc28ca50009368318bac126a522d61da79ef59d079032"} Sep 08 18:39:01.176575 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:01.176189 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-18913a0c-6d84f859df-9bj4k" event={"ID":"d3235575-cac1-402c-bc8f-c02a34352a38","Type":"ContainerStarted","Data":"84e8371cd3333e35b0ded879707ce1bf9ed8bfb706f07897e6a1624ae0e01191"} Sep 08 18:39:01.176575 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:01.176242 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-18913a0c-6d84f859df-9bj4k" Sep 08 18:39:01.177867 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:01.177842 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-18913a0c-df86496c8-vktz6" event={"ID":"cbd304f4-2c6c-4ddb-8f0d-ada5ca55b96a","Type":"ContainerStarted","Data":"f05a6a92934ee46080d5c307201a875022563ebf15734afddb5381fb1fdfdc77"} Sep 08 18:39:01.177974 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:01.177879 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-18913a0c-df86496c8-vktz6" Sep 08 18:39:01.194024 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:01.193985 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-18913a0c-6d84f859df-9bj4k" podStartSLOduration=2.193971807 podStartE2EDuration="2.193971807s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:38:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:39:01.191523088 +0000 UTC m=+923.397508487" watchObservedRunningTime="2026-09-08 18:39:01.193971807 +0000 UTC m=+923.399957203" Sep 08 18:39:01.208405 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:01.208358 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-18913a0c-df86496c8-vktz6" podStartSLOduration=2.208347201 podStartE2EDuration="2.208347201s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:38:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:39:01.205779998 +0000 UTC m=+923.411765396" watchObservedRunningTime="2026-09-08 18:39:01.208347201 +0000 UTC m=+923.414332598" Sep 08 18:39:08.726855 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:08.726822 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:39:12.186126 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:12.186092 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-18913a0c-df86496c8-vktz6" Sep 08 18:39:12.186477 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:12.186202 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-18913a0c-6d84f859df-9bj4k" Sep 08 18:39:12.241068 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:12.241037 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:14.877033 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:14.876997 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:15.579007 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:15.578979 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:16.419038 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:16.419008 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:18.525037 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:18.524983 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:19.793060 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:19.793034 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:22.509525 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:22.509463 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:22.565901 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:22.565868 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:24.939383 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:24.939350 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:25.553043 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:25.552994 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:25.578162 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:25.578126 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-0eaf99-6b47649f65-v6jj6"] Sep 08 18:39:25.578379 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:25.578354 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-0eaf99-6b47649f65-v6jj6" podUID="3e9117f7-78a4-4288-bfd9-9dbc3e7c170a" containerName="payload-processing" containerID="cri-o://2db6fabbf5866b5c3cbd4dbe95ebce17be463cc76f1aa764e53ba1b4b7a2918d" gracePeriod=30 Sep 08 18:39:25.652947 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:25.652919 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:25.745749 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:25.745718 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:25.814602 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:25.814574 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:25.827817 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:25.827793 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-0eaf99-6b47649f65-v6jj6" Sep 08 18:39:25.866424 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:25.866393 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e9117f7-78a4-4288-bfd9-9dbc3e7c170a-plugins-config-volume\") pod \"3e9117f7-78a4-4288-bfd9-9dbc3e7c170a\" (UID: \"3e9117f7-78a4-4288-bfd9-9dbc3e7c170a\") " Sep 08 18:39:25.866570 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:25.866454 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5sn9g\" (UniqueName: \"kubernetes.io/projected/3e9117f7-78a4-4288-bfd9-9dbc3e7c170a-kube-api-access-5sn9g\") pod \"3e9117f7-78a4-4288-bfd9-9dbc3e7c170a\" (UID: \"3e9117f7-78a4-4288-bfd9-9dbc3e7c170a\") " Sep 08 18:39:25.866760 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:25.866738 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e9117f7-78a4-4288-bfd9-9dbc3e7c170a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3e9117f7-78a4-4288-bfd9-9dbc3e7c170a" (UID: "3e9117f7-78a4-4288-bfd9-9dbc3e7c170a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:39:25.868586 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:25.868555 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e9117f7-78a4-4288-bfd9-9dbc3e7c170a-kube-api-access-5sn9g" (OuterVolumeSpecName: "kube-api-access-5sn9g") pod "3e9117f7-78a4-4288-bfd9-9dbc3e7c170a" (UID: "3e9117f7-78a4-4288-bfd9-9dbc3e7c170a"). InnerVolumeSpecName "kube-api-access-5sn9g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:39:25.967967 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:25.967935 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e9117f7-78a4-4288-bfd9-9dbc3e7c170a-plugins-config-volume\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:39:25.967967 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:25.967962 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5sn9g\" (UniqueName: \"kubernetes.io/projected/3e9117f7-78a4-4288-bfd9-9dbc3e7c170a-kube-api-access-5sn9g\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:39:26.093165 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:26.093073 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:26.332064 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:26.332024 2555 generic.go:356] "Generic (PLEG): container finished" podID="3e9117f7-78a4-4288-bfd9-9dbc3e7c170a" containerID="2db6fabbf5866b5c3cbd4dbe95ebce17be463cc76f1aa764e53ba1b4b7a2918d" exitCode=0 Sep 08 18:39:26.332064 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:26.332055 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-0eaf99-6b47649f65-v6jj6" event={"ID":"3e9117f7-78a4-4288-bfd9-9dbc3e7c170a","Type":"ContainerDied","Data":"2db6fabbf5866b5c3cbd4dbe95ebce17be463cc76f1aa764e53ba1b4b7a2918d"} Sep 08 18:39:26.332290 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:26.332078 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-0eaf99-6b47649f65-v6jj6" Sep 08 18:39:26.332290 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:26.332119 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-0eaf99-6b47649f65-v6jj6" event={"ID":"3e9117f7-78a4-4288-bfd9-9dbc3e7c170a","Type":"ContainerDied","Data":"2cecbb1b8f2220d3b0bd42de1fdcd4598e32f749fceb65c533f9a83402b9c628"} Sep 08 18:39:26.332290 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:26.332144 2555 scope.go:117] "RemoveContainer" containerID="2db6fabbf5866b5c3cbd4dbe95ebce17be463cc76f1aa764e53ba1b4b7a2918d" Sep 08 18:39:26.342040 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:26.342022 2555 scope.go:117] "RemoveContainer" containerID="2db6fabbf5866b5c3cbd4dbe95ebce17be463cc76f1aa764e53ba1b4b7a2918d" Sep 08 18:39:26.342290 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:39:26.342273 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2db6fabbf5866b5c3cbd4dbe95ebce17be463cc76f1aa764e53ba1b4b7a2918d\": container with ID starting with 2db6fabbf5866b5c3cbd4dbe95ebce17be463cc76f1aa764e53ba1b4b7a2918d not found: ID does not exist" containerID="2db6fabbf5866b5c3cbd4dbe95ebce17be463cc76f1aa764e53ba1b4b7a2918d" Sep 08 18:39:26.342334 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:26.342297 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2db6fabbf5866b5c3cbd4dbe95ebce17be463cc76f1aa764e53ba1b4b7a2918d"} err="failed to get container status \"2db6fabbf5866b5c3cbd4dbe95ebce17be463cc76f1aa764e53ba1b4b7a2918d\": rpc error: code = NotFound desc = could not find container \"2db6fabbf5866b5c3cbd4dbe95ebce17be463cc76f1aa764e53ba1b4b7a2918d\": container with ID starting with 2db6fabbf5866b5c3cbd4dbe95ebce17be463cc76f1aa764e53ba1b4b7a2918d not found: ID does not exist" Sep 08 18:39:26.359039 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:26.358978 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-0eaf99-6b47649f65-v6jj6"] Sep 08 18:39:26.363195 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:26.363174 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-0eaf99-6b47649f65-v6jj6"] Sep 08 18:39:28.324732 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:28.324701 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e9117f7-78a4-4288-bfd9-9dbc3e7c170a" path="/var/lib/kubelet/pods/3e9117f7-78a4-4288-bfd9-9dbc3e7c170a/volumes" Sep 08 18:39:28.937519 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:28.937483 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:29.526605 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:29.526569 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:39:29.589421 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:29.589386 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:30.160467 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:30.160431 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:30.191199 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:30.191163 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-18913a0c-6d84f859df-9bj4k"] Sep 08 18:39:30.191451 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:30.191421 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-18913a0c-6d84f859df-9bj4k" podUID="d3235575-cac1-402c-bc8f-c02a34352a38" containerName="payload-processing" containerID="cri-o://4c350d2874b051eb363fc28ca50009368318bac126a522d61da79ef59d079032" gracePeriod=30 Sep 08 18:39:30.195998 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:30.195973 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-18913a0c-df86496c8-vktz6"] Sep 08 18:39:30.196380 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:30.196347 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-18913a0c-df86496c8-vktz6" podUID="cbd304f4-2c6c-4ddb-8f0d-ada5ca55b96a" containerName="payload-pre-processing" containerID="cri-o://f05a6a92934ee46080d5c307201a875022563ebf15734afddb5381fb1fdfdc77" gracePeriod=30 Sep 08 18:39:30.233806 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:30.233781 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:30.332565 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:30.332535 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:30.357945 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:30.357906 2555 generic.go:356] "Generic (PLEG): container finished" podID="d3235575-cac1-402c-bc8f-c02a34352a38" containerID="4c350d2874b051eb363fc28ca50009368318bac126a522d61da79ef59d079032" exitCode=0 Sep 08 18:39:30.358130 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:30.358017 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-18913a0c-6d84f859df-9bj4k" event={"ID":"d3235575-cac1-402c-bc8f-c02a34352a38","Type":"ContainerDied","Data":"4c350d2874b051eb363fc28ca50009368318bac126a522d61da79ef59d079032"} Sep 08 18:39:30.360195 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:30.360170 2555 generic.go:356] "Generic (PLEG): container finished" podID="cbd304f4-2c6c-4ddb-8f0d-ada5ca55b96a" containerID="f05a6a92934ee46080d5c307201a875022563ebf15734afddb5381fb1fdfdc77" exitCode=0 Sep 08 18:39:30.360287 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:30.360246 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-18913a0c-df86496c8-vktz6" event={"ID":"cbd304f4-2c6c-4ddb-8f0d-ada5ca55b96a","Type":"ContainerDied","Data":"f05a6a92934ee46080d5c307201a875022563ebf15734afddb5381fb1fdfdc77"} Sep 08 18:39:30.423835 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:30.423732 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:30.490121 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:30.490093 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-18913a0c-6d84f859df-9bj4k" Sep 08 18:39:30.492863 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:30.492843 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-18913a0c-df86496c8-vktz6" Sep 08 18:39:30.515953 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:30.515922 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:30.613859 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:30.613829 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbdj6\" (UniqueName: \"kubernetes.io/projected/cbd304f4-2c6c-4ddb-8f0d-ada5ca55b96a-kube-api-access-bbdj6\") pod \"cbd304f4-2c6c-4ddb-8f0d-ada5ca55b96a\" (UID: \"cbd304f4-2c6c-4ddb-8f0d-ada5ca55b96a\") " Sep 08 18:39:30.614281 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:30.613932 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cbd304f4-2c6c-4ddb-8f0d-ada5ca55b96a-plugins-config-volume\") pod \"cbd304f4-2c6c-4ddb-8f0d-ada5ca55b96a\" (UID: \"cbd304f4-2c6c-4ddb-8f0d-ada5ca55b96a\") " Sep 08 18:39:30.614281 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:30.613962 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3235575-cac1-402c-bc8f-c02a34352a38-plugins-config-volume\") pod \"d3235575-cac1-402c-bc8f-c02a34352a38\" (UID: \"d3235575-cac1-402c-bc8f-c02a34352a38\") " Sep 08 18:39:30.614281 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:30.613979 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l5wxx\" (UniqueName: \"kubernetes.io/projected/d3235575-cac1-402c-bc8f-c02a34352a38-kube-api-access-l5wxx\") pod \"d3235575-cac1-402c-bc8f-c02a34352a38\" (UID: \"d3235575-cac1-402c-bc8f-c02a34352a38\") " Sep 08 18:39:30.614448 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:30.614297 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3235575-cac1-402c-bc8f-c02a34352a38-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d3235575-cac1-402c-bc8f-c02a34352a38" (UID: "d3235575-cac1-402c-bc8f-c02a34352a38"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:39:30.614448 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:30.614346 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbd304f4-2c6c-4ddb-8f0d-ada5ca55b96a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cbd304f4-2c6c-4ddb-8f0d-ada5ca55b96a" (UID: "cbd304f4-2c6c-4ddb-8f0d-ada5ca55b96a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:39:30.616161 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:30.616131 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbd304f4-2c6c-4ddb-8f0d-ada5ca55b96a-kube-api-access-bbdj6" (OuterVolumeSpecName: "kube-api-access-bbdj6") pod "cbd304f4-2c6c-4ddb-8f0d-ada5ca55b96a" (UID: "cbd304f4-2c6c-4ddb-8f0d-ada5ca55b96a"). InnerVolumeSpecName "kube-api-access-bbdj6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:39:30.616313 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:30.616279 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3235575-cac1-402c-bc8f-c02a34352a38-kube-api-access-l5wxx" (OuterVolumeSpecName: "kube-api-access-l5wxx") pod "d3235575-cac1-402c-bc8f-c02a34352a38" (UID: "d3235575-cac1-402c-bc8f-c02a34352a38"). InnerVolumeSpecName "kube-api-access-l5wxx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:39:30.715095 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:30.715013 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cbd304f4-2c6c-4ddb-8f0d-ada5ca55b96a-plugins-config-volume\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:39:30.715095 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:30.715042 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3235575-cac1-402c-bc8f-c02a34352a38-plugins-config-volume\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:39:30.715095 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:30.715052 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l5wxx\" (UniqueName: \"kubernetes.io/projected/d3235575-cac1-402c-bc8f-c02a34352a38-kube-api-access-l5wxx\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:39:30.715095 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:30.715061 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbdj6\" (UniqueName: \"kubernetes.io/projected/cbd304f4-2c6c-4ddb-8f0d-ada5ca55b96a-kube-api-access-bbdj6\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:39:31.174271 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:31.174233 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:31.281946 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:31.281914 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:31.367547 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:31.367511 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-18913a0c-6d84f859df-9bj4k" Sep 08 18:39:31.367736 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:31.367510 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-18913a0c-6d84f859df-9bj4k" event={"ID":"d3235575-cac1-402c-bc8f-c02a34352a38","Type":"ContainerDied","Data":"84e8371cd3333e35b0ded879707ce1bf9ed8bfb706f07897e6a1624ae0e01191"} Sep 08 18:39:31.367736 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:31.367642 2555 scope.go:117] "RemoveContainer" containerID="4c350d2874b051eb363fc28ca50009368318bac126a522d61da79ef59d079032" Sep 08 18:39:31.369025 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:31.368865 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-18913a0c-df86496c8-vktz6" event={"ID":"cbd304f4-2c6c-4ddb-8f0d-ada5ca55b96a","Type":"ContainerDied","Data":"8bfa19db24db270c23eea5c4e6a9d4dfcc0dd542356c8e4bbe32af990accce63"} Sep 08 18:39:31.369025 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:31.368921 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-18913a0c-df86496c8-vktz6" Sep 08 18:39:31.371823 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:31.371800 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:31.377928 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:31.377860 2555 scope.go:117] "RemoveContainer" containerID="f05a6a92934ee46080d5c307201a875022563ebf15734afddb5381fb1fdfdc77" Sep 08 18:39:31.395172 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:31.395150 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-18913a0c-df86496c8-vktz6"] Sep 08 18:39:31.402547 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:31.402524 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-18913a0c-df86496c8-vktz6"] Sep 08 18:39:31.411967 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:31.411944 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-18913a0c-6d84f859df-9bj4k"] Sep 08 18:39:31.417082 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:31.417060 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-18913a0c-6d84f859df-9bj4k"] Sep 08 18:39:32.324900 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:32.324867 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbd304f4-2c6c-4ddb-8f0d-ada5ca55b96a" path="/var/lib/kubelet/pods/cbd304f4-2c6c-4ddb-8f0d-ada5ca55b96a/volumes" Sep 08 18:39:32.325272 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:32.325214 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3235575-cac1-402c-bc8f-c02a34352a38" path="/var/lib/kubelet/pods/d3235575-cac1-402c-bc8f-c02a34352a38/volumes" Sep 08 18:39:32.742920 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:32.742839 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:37.054293 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:37.054260 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:37.283399 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:37.283365 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:37.729785 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:37.729758 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:38.050863 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:38.050830 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:38.280317 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:38.280289 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:38.834334 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:38.834264 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:38.904044 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:38.904019 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:38.982051 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:38.982026 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:39.077149 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:39.077111 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:39.176568 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:39.176500 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:39.510468 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:39.510396 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:41.460879 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:41.460837 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:41.546841 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:41.546818 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:42.584932 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:42.584889 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:42.689163 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:42.689124 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:42.783331 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:42.783292 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:43.064343 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:43.064313 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:43.134959 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:43.134923 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:39:43.172250 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:43.172220 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-c5ee49-c9b759f94-c5hn5"] Sep 08 18:39:43.172610 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:43.172598 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6aeaf180-2f92-446b-bd82-8325e7e7289b" containerName="istio-proxy" Sep 08 18:39:43.172660 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:43.172612 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="6aeaf180-2f92-446b-bd82-8325e7e7289b" containerName="istio-proxy" Sep 08 18:39:43.172660 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:43.172631 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e9117f7-78a4-4288-bfd9-9dbc3e7c170a" containerName="payload-processing" Sep 08 18:39:43.172660 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:43.172637 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e9117f7-78a4-4288-bfd9-9dbc3e7c170a" containerName="payload-processing" Sep 08 18:39:43.172660 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:43.172643 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3235575-cac1-402c-bc8f-c02a34352a38" containerName="payload-processing" Sep 08 18:39:43.172660 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:43.172650 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3235575-cac1-402c-bc8f-c02a34352a38" containerName="payload-processing" Sep 08 18:39:43.172832 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:43.172670 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbd304f4-2c6c-4ddb-8f0d-ada5ca55b96a" containerName="payload-pre-processing" Sep 08 18:39:43.172832 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:43.172675 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbd304f4-2c6c-4ddb-8f0d-ada5ca55b96a" containerName="payload-pre-processing" Sep 08 18:39:43.172832 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:43.172749 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e9117f7-78a4-4288-bfd9-9dbc3e7c170a" containerName="payload-processing" Sep 08 18:39:43.172832 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:43.172757 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3235575-cac1-402c-bc8f-c02a34352a38" containerName="payload-processing" Sep 08 18:39:43.172832 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:43.172764 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbd304f4-2c6c-4ddb-8f0d-ada5ca55b96a" containerName="payload-pre-processing" Sep 08 18:39:43.177157 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:43.177142 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c5ee49-c9b759f94-c5hn5" Sep 08 18:39:43.179586 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:43.179564 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-c5ee49\"" Sep 08 18:39:43.179586 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:43.179566 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-c5ee49-dockercfg-rkblk\"" Sep 08 18:39:43.187627 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:43.187608 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-c5ee49-c9b759f94-c5hn5"] Sep 08 18:39:43.323841 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:43.323757 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:39:43.335250 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:43.335229 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9qxg\" (UniqueName: \"kubernetes.io/projected/3faf5860-09da-4400-b521-3dc585ed751b-kube-api-access-t9qxg\") pod \"payload-pre-processing-e2e-explicit-c5ee49-c9b759f94-c5hn5\" (UID: \"3faf5860-09da-4400-b521-3dc585ed751b\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-c5ee49-c9b759f94-c5hn5" Sep 08 18:39:43.335330 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:43.335313 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3faf5860-09da-4400-b521-3dc585ed751b-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-c5ee49-c9b759f94-c5hn5\" (UID: \"3faf5860-09da-4400-b521-3dc585ed751b\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-c5ee49-c9b759f94-c5hn5" Sep 08 18:39:43.436488 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:43.436460 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9qxg\" (UniqueName: \"kubernetes.io/projected/3faf5860-09da-4400-b521-3dc585ed751b-kube-api-access-t9qxg\") pod \"payload-pre-processing-e2e-explicit-c5ee49-c9b759f94-c5hn5\" (UID: \"3faf5860-09da-4400-b521-3dc585ed751b\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-c5ee49-c9b759f94-c5hn5" Sep 08 18:39:43.436658 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:43.436517 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3faf5860-09da-4400-b521-3dc585ed751b-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-c5ee49-c9b759f94-c5hn5\" (UID: \"3faf5860-09da-4400-b521-3dc585ed751b\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-c5ee49-c9b759f94-c5hn5" Sep 08 18:39:43.437273 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:43.437242 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3faf5860-09da-4400-b521-3dc585ed751b-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-c5ee49-c9b759f94-c5hn5\" (UID: \"3faf5860-09da-4400-b521-3dc585ed751b\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-c5ee49-c9b759f94-c5hn5" Sep 08 18:39:43.445421 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:43.445402 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9qxg\" (UniqueName: \"kubernetes.io/projected/3faf5860-09da-4400-b521-3dc585ed751b-kube-api-access-t9qxg\") pod \"payload-pre-processing-e2e-explicit-c5ee49-c9b759f94-c5hn5\" (UID: \"3faf5860-09da-4400-b521-3dc585ed751b\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-c5ee49-c9b759f94-c5hn5" Sep 08 18:39:43.488020 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:43.487990 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c5ee49-c9b759f94-c5hn5" Sep 08 18:39:43.759586 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:43.759561 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-c5ee49-c9b759f94-c5hn5"] Sep 08 18:39:43.760647 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:39:43.760621 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3faf5860_09da_4400_b521_3dc585ed751b.slice/crio-dfbe4a876290a975ba81bce8f204c0c2897227d85f8e62673e07f3622e4600d2 WatchSource:0}: Error finding container dfbe4a876290a975ba81bce8f204c0c2897227d85f8e62673e07f3622e4600d2: Status 404 returned error can't find the container with id dfbe4a876290a975ba81bce8f204c0c2897227d85f8e62673e07f3622e4600d2 Sep 08 18:39:44.444862 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:44.444827 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c5ee49-c9b759f94-c5hn5" event={"ID":"3faf5860-09da-4400-b521-3dc585ed751b","Type":"ContainerStarted","Data":"f0f1ec4db6d574c092a7fbb1453d2561fa7c2ac0fee622b8a54f747819e5dcc8"} Sep 08 18:39:44.445072 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:44.444882 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c5ee49-c9b759f94-c5hn5" event={"ID":"3faf5860-09da-4400-b521-3dc585ed751b","Type":"ContainerStarted","Data":"dfbe4a876290a975ba81bce8f204c0c2897227d85f8e62673e07f3622e4600d2"} Sep 08 18:39:44.445072 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:44.444909 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c5ee49-c9b759f94-c5hn5" Sep 08 18:39:44.460492 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:44.460443 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c5ee49-c9b759f94-c5hn5" podStartSLOduration=1.4604286530000001 podStartE2EDuration="1.460428653s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:39:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:39:44.459015838 +0000 UTC m=+966.665001261" watchObservedRunningTime="2026-09-08 18:39:44.460428653 +0000 UTC m=+966.666414050" Sep 08 18:39:55.452448 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:55.452414 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c5ee49-c9b759f94-c5hn5" Sep 08 18:39:58.120702 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:39:58.120655 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:40:04.675154 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:04.675077 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:40:05.514804 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:05.514766 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:40:06.362212 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:06.362174 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:40:07.529554 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:07.529502 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:40:11.472673 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:11.472639 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:40:11.534914 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:11.534863 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:40:11.610984 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:11.610933 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:40:22.233025 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:22.232988 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:40:27.491579 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:27.491517 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:40:33.463314 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:33.463281 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:40:33.532158 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:33.532106 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:40:33.608453 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:33.608419 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:40:36.921279 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:36.921245 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:40:42.044419 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:42.044387 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:40:44.140395 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:44.140365 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:40:44.813457 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:44.813425 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:40:44.943552 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:44.943518 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c5ee49-ztltb"] Sep 08 18:40:44.944809 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:44.944785 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:40:44.947374 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:44.947361 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c5ee49-ztltb" Sep 08 18:40:44.949934 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:44.949916 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g6q8h\"" Sep 08 18:40:44.950201 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:44.950187 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 08 18:40:44.954972 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:44.954698 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c5ee49-ztltb"] Sep 08 18:40:45.017196 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:45.017168 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/02d9f09c-4935-4584-bad7-22f686cf6be3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-c5ee49-ztltb\" (UID: \"02d9f09c-4935-4584-bad7-22f686cf6be3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c5ee49-ztltb" Sep 08 18:40:45.118810 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:45.118678 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/02d9f09c-4935-4584-bad7-22f686cf6be3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-c5ee49-ztltb\" (UID: \"02d9f09c-4935-4584-bad7-22f686cf6be3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c5ee49-ztltb" Sep 08 18:40:45.119289 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:45.119269 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/02d9f09c-4935-4584-bad7-22f686cf6be3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-c5ee49-ztltb\" (UID: \"02d9f09c-4935-4584-bad7-22f686cf6be3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c5ee49-ztltb" Sep 08 18:40:45.259343 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:45.259301 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c5ee49-ztltb" Sep 08 18:40:45.529331 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:45.529238 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c5ee49-ztltb"] Sep 08 18:40:45.534049 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:40:45.534024 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02d9f09c_4935_4584_bad7_22f686cf6be3.slice/crio-a89b452bac0ac0194349983ffa5012794e1a44d855f5c14cd9018ab1330a2d2f WatchSource:0}: Error finding container a89b452bac0ac0194349983ffa5012794e1a44d855f5c14cd9018ab1330a2d2f: Status 404 returned error can't find the container with id a89b452bac0ac0194349983ffa5012794e1a44d855f5c14cd9018ab1330a2d2f Sep 08 18:40:45.794815 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:45.794787 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c5ee49-ztltb" event={"ID":"02d9f09c-4935-4584-bad7-22f686cf6be3","Type":"ContainerStarted","Data":"a89b452bac0ac0194349983ffa5012794e1a44d855f5c14cd9018ab1330a2d2f"} Sep 08 18:40:46.803044 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:46.803012 2555 generic.go:356] "Generic (PLEG): container finished" podID="02d9f09c-4935-4584-bad7-22f686cf6be3" containerID="27bdb54df60cc0cb0f376acf2a02222ee56735481331210974c1106302c0fce2" exitCode=0 Sep 08 18:40:46.803444 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:46.803111 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c5ee49-ztltb" event={"ID":"02d9f09c-4935-4584-bad7-22f686cf6be3","Type":"ContainerDied","Data":"27bdb54df60cc0cb0f376acf2a02222ee56735481331210974c1106302c0fce2"} Sep 08 18:40:47.944548 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:47.944519 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c5ee49-ztltb" Sep 08 18:40:48.044567 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:48.044532 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/02d9f09c-4935-4584-bad7-22f686cf6be3-maas-api-service-ca\") pod \"02d9f09c-4935-4584-bad7-22f686cf6be3\" (UID: \"02d9f09c-4935-4584-bad7-22f686cf6be3\") " Sep 08 18:40:48.044923 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:48.044902 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02d9f09c-4935-4584-bad7-22f686cf6be3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "02d9f09c-4935-4584-bad7-22f686cf6be3" (UID: "02d9f09c-4935-4584-bad7-22f686cf6be3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:40:48.146058 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:48.145978 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/02d9f09c-4935-4584-bad7-22f686cf6be3-maas-api-service-ca\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:40:48.816113 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:48.816083 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c5ee49-ztltb" Sep 08 18:40:48.816291 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:48.816113 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c5ee49-ztltb" event={"ID":"02d9f09c-4935-4584-bad7-22f686cf6be3","Type":"ContainerDied","Data":"a89b452bac0ac0194349983ffa5012794e1a44d855f5c14cd9018ab1330a2d2f"} Sep 08 18:40:48.816291 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:48.816159 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a89b452bac0ac0194349983ffa5012794e1a44d855f5c14cd9018ab1330a2d2f" Sep 08 18:40:50.635983 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:50.635944 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:40:50.739168 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:50.739133 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:40:50.835344 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:50.835308 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:40:50.935446 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:50.935361 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:40:51.568782 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:51.568747 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:40:51.651486 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:51.651454 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:40:51.751657 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:51.751624 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:40:53.222843 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:53.222807 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:40:55.418176 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:55.418148 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:40:55.438220 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:55.438186 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-c5ee49-c9b759f94-c5hn5"] Sep 08 18:40:55.438711 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:55.438524 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c5ee49-c9b759f94-c5hn5" podUID="3faf5860-09da-4400-b521-3dc585ed751b" containerName="payload-pre-processing" containerID="cri-o://f0f1ec4db6d574c092a7fbb1453d2561fa7c2ac0fee622b8a54f747819e5dcc8" gracePeriod=30 Sep 08 18:40:55.451130 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:55.451093 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c5ee49-c9b759f94-c5hn5" podUID="3faf5860-09da-4400-b521-3dc585ed751b" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.59:9004: connect: connection refused" Sep 08 18:40:55.516499 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:55.516468 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:40:55.593093 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:55.593060 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c5ee49-ztltb"] Sep 08 18:40:55.594734 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:55.594708 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:40:55.595824 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:55.595802 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-c5ee49-ztltb"] Sep 08 18:40:55.655321 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:55.655288 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:40:55.697418 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:55.697393 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c5ee49-c9b759f94-c5hn5" Sep 08 18:40:55.823720 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:55.823653 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3faf5860-09da-4400-b521-3dc585ed751b-plugins-config-volume\") pod \"3faf5860-09da-4400-b521-3dc585ed751b\" (UID: \"3faf5860-09da-4400-b521-3dc585ed751b\") " Sep 08 18:40:55.823930 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:55.823810 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9qxg\" (UniqueName: \"kubernetes.io/projected/3faf5860-09da-4400-b521-3dc585ed751b-kube-api-access-t9qxg\") pod \"3faf5860-09da-4400-b521-3dc585ed751b\" (UID: \"3faf5860-09da-4400-b521-3dc585ed751b\") " Sep 08 18:40:55.824060 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:55.824034 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3faf5860-09da-4400-b521-3dc585ed751b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3faf5860-09da-4400-b521-3dc585ed751b" (UID: "3faf5860-09da-4400-b521-3dc585ed751b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:40:55.826046 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:55.826026 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3faf5860-09da-4400-b521-3dc585ed751b-kube-api-access-t9qxg" (OuterVolumeSpecName: "kube-api-access-t9qxg") pod "3faf5860-09da-4400-b521-3dc585ed751b" (UID: "3faf5860-09da-4400-b521-3dc585ed751b"). InnerVolumeSpecName "kube-api-access-t9qxg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:40:55.858160 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:55.858124 2555 generic.go:356] "Generic (PLEG): container finished" podID="3faf5860-09da-4400-b521-3dc585ed751b" containerID="f0f1ec4db6d574c092a7fbb1453d2561fa7c2ac0fee622b8a54f747819e5dcc8" exitCode=0 Sep 08 18:40:55.858339 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:55.858174 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c5ee49-c9b759f94-c5hn5" event={"ID":"3faf5860-09da-4400-b521-3dc585ed751b","Type":"ContainerDied","Data":"f0f1ec4db6d574c092a7fbb1453d2561fa7c2ac0fee622b8a54f747819e5dcc8"} Sep 08 18:40:55.858339 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:55.858213 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c5ee49-c9b759f94-c5hn5" event={"ID":"3faf5860-09da-4400-b521-3dc585ed751b","Type":"ContainerDied","Data":"dfbe4a876290a975ba81bce8f204c0c2897227d85f8e62673e07f3622e4600d2"} Sep 08 18:40:55.858339 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:55.858174 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c5ee49-c9b759f94-c5hn5" Sep 08 18:40:55.858339 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:55.858234 2555 scope.go:117] "RemoveContainer" containerID="f0f1ec4db6d574c092a7fbb1453d2561fa7c2ac0fee622b8a54f747819e5dcc8" Sep 08 18:40:55.867711 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:55.867669 2555 scope.go:117] "RemoveContainer" containerID="f0f1ec4db6d574c092a7fbb1453d2561fa7c2ac0fee622b8a54f747819e5dcc8" Sep 08 18:40:55.868025 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:40:55.868004 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0f1ec4db6d574c092a7fbb1453d2561fa7c2ac0fee622b8a54f747819e5dcc8\": container with ID starting with f0f1ec4db6d574c092a7fbb1453d2561fa7c2ac0fee622b8a54f747819e5dcc8 not found: ID does not exist" containerID="f0f1ec4db6d574c092a7fbb1453d2561fa7c2ac0fee622b8a54f747819e5dcc8" Sep 08 18:40:55.868079 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:55.868035 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0f1ec4db6d574c092a7fbb1453d2561fa7c2ac0fee622b8a54f747819e5dcc8"} err="failed to get container status \"f0f1ec4db6d574c092a7fbb1453d2561fa7c2ac0fee622b8a54f747819e5dcc8\": rpc error: code = NotFound desc = could not find container \"f0f1ec4db6d574c092a7fbb1453d2561fa7c2ac0fee622b8a54f747819e5dcc8\": container with ID starting with f0f1ec4db6d574c092a7fbb1453d2561fa7c2ac0fee622b8a54f747819e5dcc8 not found: ID does not exist" Sep 08 18:40:55.879365 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:55.879327 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-c5ee49-c9b759f94-c5hn5"] Sep 08 18:40:55.883047 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:55.883024 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-c5ee49-c9b759f94-c5hn5"] Sep 08 18:40:55.924888 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:55.924853 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9qxg\" (UniqueName: \"kubernetes.io/projected/3faf5860-09da-4400-b521-3dc585ed751b-kube-api-access-t9qxg\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:40:55.924888 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:55.924882 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3faf5860-09da-4400-b521-3dc585ed751b-plugins-config-volume\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:40:55.948610 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:55.948519 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:40:56.324732 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:56.324678 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02d9f09c-4935-4584-bad7-22f686cf6be3" path="/var/lib/kubelet/pods/02d9f09c-4935-4584-bad7-22f686cf6be3/volumes" Sep 08 18:40:56.325079 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:56.325066 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3faf5860-09da-4400-b521-3dc585ed751b" path="/var/lib/kubelet/pods/3faf5860-09da-4400-b521-3dc585ed751b/volumes" Sep 08 18:40:56.932069 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:56.932029 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:40:59.304445 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:59.304412 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:40:59.371467 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:59.371436 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:40:59.957975 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:40:59.957940 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:00.999163 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:00.999132 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:01.941627 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:01.941594 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:02.054945 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:02.054904 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:02.245323 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:02.245218 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:02.618319 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:02.618280 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:02.920781 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:02.920679 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:03.374516 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:03.374479 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:04.311732 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:04.311703 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:04.407202 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:04.407164 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:04.509530 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:04.509477 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:05.767747 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:05.767716 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:05.866077 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:05.866038 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:05.997217 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:05.997186 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:06.340756 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.340723 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj"] Sep 08 18:41:06.341111 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.341097 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3faf5860-09da-4400-b521-3dc585ed751b" containerName="payload-pre-processing" Sep 08 18:41:06.341111 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.341110 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="3faf5860-09da-4400-b521-3dc585ed751b" containerName="payload-pre-processing" Sep 08 18:41:06.341243 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.341133 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02d9f09c-4935-4584-bad7-22f686cf6be3" containerName="revoke-keys" Sep 08 18:41:06.341243 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.341138 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="02d9f09c-4935-4584-bad7-22f686cf6be3" containerName="revoke-keys" Sep 08 18:41:06.341243 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.341208 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="02d9f09c-4935-4584-bad7-22f686cf6be3" containerName="revoke-keys" Sep 08 18:41:06.341243 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.341218 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="3faf5860-09da-4400-b521-3dc585ed751b" containerName="payload-pre-processing" Sep 08 18:41:06.344358 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.344335 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:41:06.346981 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.346961 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-2e25b35c-openshift-default-dockercfg-2m894\"" Sep 08 18:41:06.359359 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.359338 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj"] Sep 08 18:41:06.388263 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.388230 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:06.489901 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.489874 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:06.531117 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.531085 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ec70540-6806-4667-aed6-c4891a8ae835-istio-token\") pod \"e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj\" (UID: \"3ec70540-6806-4667-aed6-c4891a8ae835\") " pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:41:06.531117 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.531121 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ec70540-6806-4667-aed6-c4891a8ae835-workload-certs\") pod \"e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj\" (UID: \"3ec70540-6806-4667-aed6-c4891a8ae835\") " pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:41:06.531357 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.531159 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wd278\" (UniqueName: \"kubernetes.io/projected/3ec70540-6806-4667-aed6-c4891a8ae835-kube-api-access-wd278\") pod \"e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj\" (UID: \"3ec70540-6806-4667-aed6-c4891a8ae835\") " pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:41:06.531357 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.531234 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ec70540-6806-4667-aed6-c4891a8ae835-istio-envoy\") pod \"e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj\" (UID: \"3ec70540-6806-4667-aed6-c4891a8ae835\") " pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:41:06.531357 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.531262 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ec70540-6806-4667-aed6-c4891a8ae835-istiod-ca-cert\") pod \"e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj\" (UID: \"3ec70540-6806-4667-aed6-c4891a8ae835\") " pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:41:06.531357 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.531287 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ec70540-6806-4667-aed6-c4891a8ae835-credential-socket\") pod \"e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj\" (UID: \"3ec70540-6806-4667-aed6-c4891a8ae835\") " pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:41:06.531357 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.531308 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ec70540-6806-4667-aed6-c4891a8ae835-istio-data\") pod \"e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj\" (UID: \"3ec70540-6806-4667-aed6-c4891a8ae835\") " pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:41:06.531545 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.531359 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ec70540-6806-4667-aed6-c4891a8ae835-workload-socket\") pod \"e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj\" (UID: \"3ec70540-6806-4667-aed6-c4891a8ae835\") " pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:41:06.531545 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.531376 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ec70540-6806-4667-aed6-c4891a8ae835-istio-podinfo\") pod \"e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj\" (UID: \"3ec70540-6806-4667-aed6-c4891a8ae835\") " pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:41:06.632548 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.632456 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ec70540-6806-4667-aed6-c4891a8ae835-workload-socket\") pod \"e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj\" (UID: \"3ec70540-6806-4667-aed6-c4891a8ae835\") " pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:41:06.632548 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.632496 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ec70540-6806-4667-aed6-c4891a8ae835-istio-podinfo\") pod \"e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj\" (UID: \"3ec70540-6806-4667-aed6-c4891a8ae835\") " pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:41:06.632548 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.632547 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ec70540-6806-4667-aed6-c4891a8ae835-istio-token\") pod \"e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj\" (UID: \"3ec70540-6806-4667-aed6-c4891a8ae835\") " pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:41:06.632867 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.632564 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ec70540-6806-4667-aed6-c4891a8ae835-workload-certs\") pod \"e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj\" (UID: \"3ec70540-6806-4667-aed6-c4891a8ae835\") " pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:41:06.632867 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.632722 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd278\" (UniqueName: \"kubernetes.io/projected/3ec70540-6806-4667-aed6-c4891a8ae835-kube-api-access-wd278\") pod \"e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj\" (UID: \"3ec70540-6806-4667-aed6-c4891a8ae835\") " pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:41:06.632867 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.632766 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ec70540-6806-4667-aed6-c4891a8ae835-istio-envoy\") pod \"e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj\" (UID: \"3ec70540-6806-4667-aed6-c4891a8ae835\") " pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:41:06.632867 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.632800 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ec70540-6806-4667-aed6-c4891a8ae835-istiod-ca-cert\") pod \"e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj\" (UID: \"3ec70540-6806-4667-aed6-c4891a8ae835\") " pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:41:06.632867 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.632835 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ec70540-6806-4667-aed6-c4891a8ae835-credential-socket\") pod \"e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj\" (UID: \"3ec70540-6806-4667-aed6-c4891a8ae835\") " pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:41:06.632867 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.632862 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ec70540-6806-4667-aed6-c4891a8ae835-istio-data\") pod \"e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj\" (UID: \"3ec70540-6806-4667-aed6-c4891a8ae835\") " pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:41:06.633163 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.632931 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ec70540-6806-4667-aed6-c4891a8ae835-workload-socket\") pod \"e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj\" (UID: \"3ec70540-6806-4667-aed6-c4891a8ae835\") " pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:41:06.633163 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.632954 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ec70540-6806-4667-aed6-c4891a8ae835-workload-certs\") pod \"e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj\" (UID: \"3ec70540-6806-4667-aed6-c4891a8ae835\") " pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:41:06.633267 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.633183 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ec70540-6806-4667-aed6-c4891a8ae835-credential-socket\") pod \"e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj\" (UID: \"3ec70540-6806-4667-aed6-c4891a8ae835\") " pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:41:06.633267 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.633240 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ec70540-6806-4667-aed6-c4891a8ae835-istio-data\") pod \"e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj\" (UID: \"3ec70540-6806-4667-aed6-c4891a8ae835\") " pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:41:06.633601 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.633579 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ec70540-6806-4667-aed6-c4891a8ae835-istiod-ca-cert\") pod \"e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj\" (UID: \"3ec70540-6806-4667-aed6-c4891a8ae835\") " pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:41:06.635524 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.635499 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ec70540-6806-4667-aed6-c4891a8ae835-istio-envoy\") pod \"e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj\" (UID: \"3ec70540-6806-4667-aed6-c4891a8ae835\") " pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:41:06.635983 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.635968 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ec70540-6806-4667-aed6-c4891a8ae835-istio-podinfo\") pod \"e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj\" (UID: \"3ec70540-6806-4667-aed6-c4891a8ae835\") " pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:41:06.640720 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.640678 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd278\" (UniqueName: \"kubernetes.io/projected/3ec70540-6806-4667-aed6-c4891a8ae835-kube-api-access-wd278\") pod \"e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj\" (UID: \"3ec70540-6806-4667-aed6-c4891a8ae835\") " pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:41:06.640803 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.640750 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ec70540-6806-4667-aed6-c4891a8ae835-istio-token\") pod \"e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj\" (UID: \"3ec70540-6806-4667-aed6-c4891a8ae835\") " pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:41:06.654810 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.654790 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:41:06.930648 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.930620 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj"] Sep 08 18:41:06.932384 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:41:06.932356 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ec70540_6806_4667_aed6_c4891a8ae835.slice/crio-ce7afc125cd4da3c6a638198de8c11800b14906819a19ed07a8d2c54a27f1b58 WatchSource:0}: Error finding container ce7afc125cd4da3c6a638198de8c11800b14906819a19ed07a8d2c54a27f1b58: Status 404 returned error can't find the container with id ce7afc125cd4da3c6a638198de8c11800b14906819a19ed07a8d2c54a27f1b58 Sep 08 18:41:06.934473 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.934445 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 18:41:06.934540 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.934512 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 18:41:06.934577 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:06.934542 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 18:41:07.935616 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:07.935586 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" event={"ID":"3ec70540-6806-4667-aed6-c4891a8ae835","Type":"ContainerStarted","Data":"e069074cec3ece1c7f9e3a542fc2dfc6e2218b3cea720d46b91f211fb8b4e583"} Sep 08 18:41:07.936014 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:07.935639 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" event={"ID":"3ec70540-6806-4667-aed6-c4891a8ae835","Type":"ContainerStarted","Data":"ce7afc125cd4da3c6a638198de8c11800b14906819a19ed07a8d2c54a27f1b58"} Sep 08 18:41:07.959264 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:07.959198 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" podStartSLOduration=1.9591766879999999 podStartE2EDuration="1.959176688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:41:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:41:07.954409945 +0000 UTC m=+1050.160395344" watchObservedRunningTime="2026-09-08 18:41:07.959176688 +0000 UTC m=+1050.165162085" Sep 08 18:41:08.655778 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:08.655736 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:41:08.661263 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:08.661235 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:41:08.941371 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:08.941287 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:41:08.942245 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:08.942227 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:41:09.893088 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:09.893049 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:09.994302 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:09.994267 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:11.157097 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:11.157063 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:11.258620 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:11.258585 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:11.509561 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:11.509450 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:11.616990 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:11.616955 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:11.712742 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:11.712707 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:11.717773 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:11.717750 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2e25b35c-67d478989d-c96tj"] Sep 08 18:41:11.722928 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:11.722898 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2e25b35c-67d478989d-c96tj" Sep 08 18:41:11.726385 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:11.726363 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-2e25b35c-dockercfg-wr4rw\"" Sep 08 18:41:11.726540 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:11.726403 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-2e25b35c\"" Sep 08 18:41:11.732757 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:11.732728 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2e25b35c-67d478989d-c96tj"] Sep 08 18:41:11.773983 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:11.773885 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5njm\" (UniqueName: \"kubernetes.io/projected/0a022862-8679-4dbf-b8a5-d1e4496ed6be-kube-api-access-h5njm\") pod \"payload-pre-processing-e2e-mt-2e25b35c-67d478989d-c96tj\" (UID: \"0a022862-8679-4dbf-b8a5-d1e4496ed6be\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2e25b35c-67d478989d-c96tj" Sep 08 18:41:11.774128 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:11.773996 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a022862-8679-4dbf-b8a5-d1e4496ed6be-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-2e25b35c-67d478989d-c96tj\" (UID: \"0a022862-8679-4dbf-b8a5-d1e4496ed6be\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2e25b35c-67d478989d-c96tj" Sep 08 18:41:11.875398 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:11.875362 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5njm\" (UniqueName: \"kubernetes.io/projected/0a022862-8679-4dbf-b8a5-d1e4496ed6be-kube-api-access-h5njm\") pod \"payload-pre-processing-e2e-mt-2e25b35c-67d478989d-c96tj\" (UID: \"0a022862-8679-4dbf-b8a5-d1e4496ed6be\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2e25b35c-67d478989d-c96tj" Sep 08 18:41:11.875580 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:11.875437 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a022862-8679-4dbf-b8a5-d1e4496ed6be-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-2e25b35c-67d478989d-c96tj\" (UID: \"0a022862-8679-4dbf-b8a5-d1e4496ed6be\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2e25b35c-67d478989d-c96tj" Sep 08 18:41:11.876158 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:11.876137 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a022862-8679-4dbf-b8a5-d1e4496ed6be-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-2e25b35c-67d478989d-c96tj\" (UID: \"0a022862-8679-4dbf-b8a5-d1e4496ed6be\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2e25b35c-67d478989d-c96tj" Sep 08 18:41:11.882877 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:11.882856 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5njm\" (UniqueName: \"kubernetes.io/projected/0a022862-8679-4dbf-b8a5-d1e4496ed6be-kube-api-access-h5njm\") pod \"payload-pre-processing-e2e-mt-2e25b35c-67d478989d-c96tj\" (UID: \"0a022862-8679-4dbf-b8a5-d1e4496ed6be\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2e25b35c-67d478989d-c96tj" Sep 08 18:41:12.038151 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:12.038122 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2e25b35c-67d478989d-c96tj" Sep 08 18:41:12.313375 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:12.313347 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2e25b35c-67d478989d-c96tj"] Sep 08 18:41:12.314444 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:41:12.314420 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a022862_8679_4dbf_b8a5_d1e4496ed6be.slice/crio-aef43c6809e594a6a682e2f014fff03b62180d4cd6ed8c174bc158433e6dbfdb WatchSource:0}: Error finding container aef43c6809e594a6a682e2f014fff03b62180d4cd6ed8c174bc158433e6dbfdb: Status 404 returned error can't find the container with id aef43c6809e594a6a682e2f014fff03b62180d4cd6ed8c174bc158433e6dbfdb Sep 08 18:41:12.603859 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:12.603761 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:12.963902 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:12.963810 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2e25b35c-67d478989d-c96tj" event={"ID":"0a022862-8679-4dbf-b8a5-d1e4496ed6be","Type":"ContainerStarted","Data":"247b8cc2cef468119d49b7ec69a576288885351d3dd6b8d39c3eee7d59aabb01"} Sep 08 18:41:12.963902 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:12.963864 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2e25b35c-67d478989d-c96tj" event={"ID":"0a022862-8679-4dbf-b8a5-d1e4496ed6be","Type":"ContainerStarted","Data":"aef43c6809e594a6a682e2f014fff03b62180d4cd6ed8c174bc158433e6dbfdb"} Sep 08 18:41:12.964127 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:12.963928 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-2e25b35c-67d478989d-c96tj" Sep 08 18:41:12.992789 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:12.992734 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-2e25b35c-67d478989d-c96tj" podStartSLOduration=1.992718666 podStartE2EDuration="1.992718666s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:41:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:41:12.991148565 +0000 UTC m=+1055.197133962" watchObservedRunningTime="2026-09-08 18:41:12.992718666 +0000 UTC m=+1055.198704062" Sep 08 18:41:15.226445 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:15.226405 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:41:17.014283 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:17.014179 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:17.734212 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:17.734179 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:17.834504 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:17.834476 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:18.034724 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:18.034608 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:18.132820 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:18.132790 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:18.350722 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:18.350654 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:18.459152 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:18.459120 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:18.818851 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:18.818768 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:19.054590 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:19.054562 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:23.540645 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:23.540604 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:23.972714 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:23.972608 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-2e25b35c-67d478989d-c96tj" Sep 08 18:41:24.588414 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:24.587246 2555 patch_prober.go:28] interesting pod/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": read tcp 10.132.0.2:41956->10.132.0.61:15021: read: connection reset by peer" start-of-body= Sep 08 18:41:24.588414 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:24.587320 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" podUID="3ec70540-6806-4667-aed6-c4891a8ae835" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": read tcp 10.132.0.2:41956->10.132.0.61:15021: read: connection reset by peer" Sep 08 18:41:25.064563 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:25.064534 2555 generic.go:356] "Generic (PLEG): container finished" podID="3ec70540-6806-4667-aed6-c4891a8ae835" containerID="e069074cec3ece1c7f9e3a542fc2dfc6e2218b3cea720d46b91f211fb8b4e583" exitCode=0 Sep 08 18:41:25.064764 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:25.064615 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" event={"ID":"3ec70540-6806-4667-aed6-c4891a8ae835","Type":"ContainerDied","Data":"e069074cec3ece1c7f9e3a542fc2dfc6e2218b3cea720d46b91f211fb8b4e583"} Sep 08 18:41:25.065076 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:25.065062 2555 scope.go:117] "RemoveContainer" containerID="e069074cec3ece1c7f9e3a542fc2dfc6e2218b3cea720d46b91f211fb8b4e583" Sep 08 18:41:25.066149 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:25.066124 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 18:41:25.066243 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:25.066201 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 18:41:25.066304 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:25.066244 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 18:41:26.072545 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:26.072508 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" event={"ID":"3ec70540-6806-4667-aed6-c4891a8ae835","Type":"ContainerStarted","Data":"b20ac0b79e49cb8520bcd854a4da63e89dab42dc8ed10e7daf53d8a35809318a"} Sep 08 18:41:26.654987 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:26.654950 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:41:26.655186 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:26.655164 2555 patch_prober.go:28] interesting pod/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Sep 08 18:41:26.655235 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:26.655214 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" podUID="3ec70540-6806-4667-aed6-c4891a8ae835" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Sep 08 18:41:27.339412 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:27.339373 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:41:27.655429 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:27.655335 2555 patch_prober.go:28] interesting pod/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Sep 08 18:41:27.655429 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:27.655392 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" podUID="3ec70540-6806-4667-aed6-c4891a8ae835" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Sep 08 18:41:28.656104 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:28.656069 2555 patch_prober.go:28] interesting pod/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Sep 08 18:41:28.656481 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:28.656125 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" podUID="3ec70540-6806-4667-aed6-c4891a8ae835" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Sep 08 18:41:29.356271 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:29.356192 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:29.450410 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:29.450379 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:29.534163 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:29.534132 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:29.610390 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:29.610292 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:29.656119 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:29.656070 2555 patch_prober.go:28] interesting pod/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Sep 08 18:41:29.656482 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:29.656151 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" podUID="3ec70540-6806-4667-aed6-c4891a8ae835" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Sep 08 18:41:29.967519 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:29.967428 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:30.655366 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:30.655327 2555 patch_prober.go:28] interesting pod/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Sep 08 18:41:30.655560 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:30.655393 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" podUID="3ec70540-6806-4667-aed6-c4891a8ae835" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Sep 08 18:41:31.659287 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:31.659253 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:41:31.659675 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:31.659637 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:41:31.660207 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:31.660193 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:41:33.062884 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:33.062850 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:34.226210 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:34.226172 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:41:34.905662 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:34.905628 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:35.611768 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:35.611735 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:35.713772 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:35.713741 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:36.082478 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:36.082445 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:36.409713 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:36.409600 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:36.510097 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:36.510068 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:36.664434 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:36.664347 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:37.138255 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:37.138220 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:41.992193 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:41.992158 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:42.059943 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:42.059913 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:42.645443 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:42.645409 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:47.243133 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:47.243098 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:47.783436 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:47.783399 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:47.813021 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:47.812982 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-15d796-5c8bd9fb99-2zv2v"] Sep 08 18:41:47.813632 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:47.813595 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-15d796-5c8bd9fb99-2zv2v" podUID="4822efcb-f15f-4857-b882-774b0baec527" containerName="payload-processing" containerID="cri-o://442291376e5bd98cc82f1abebe1e7731665244be47d11ef881e803356c571a79" gracePeriod=30 Sep 08 18:41:47.819471 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:47.819281 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-15d796-668bc7c5d-6m7d9"] Sep 08 18:41:47.819606 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:47.819538 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-15d796-668bc7c5d-6m7d9" podUID="241061eb-fa90-45a1-ae52-13730bf5c646" containerName="payload-pre-processing" containerID="cri-o://9cca1fbfdcea09d33163af2fc49eb250038b3e786a9861ede59906d77c5c0b19" gracePeriod=30 Sep 08 18:41:47.855018 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:47.854982 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:47.914072 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:47.914032 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:47.975803 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:47.975769 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:48.097862 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:48.097836 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-15d796-5c8bd9fb99-2zv2v" Sep 08 18:41:48.100846 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:48.100823 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-15d796-668bc7c5d-6m7d9" Sep 08 18:41:48.210205 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:48.210170 2555 generic.go:356] "Generic (PLEG): container finished" podID="4822efcb-f15f-4857-b882-774b0baec527" containerID="442291376e5bd98cc82f1abebe1e7731665244be47d11ef881e803356c571a79" exitCode=0 Sep 08 18:41:48.210383 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:48.210215 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-15d796-5c8bd9fb99-2zv2v" Sep 08 18:41:48.210383 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:48.210259 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-15d796-5c8bd9fb99-2zv2v" event={"ID":"4822efcb-f15f-4857-b882-774b0baec527","Type":"ContainerDied","Data":"442291376e5bd98cc82f1abebe1e7731665244be47d11ef881e803356c571a79"} Sep 08 18:41:48.210383 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:48.210311 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-15d796-5c8bd9fb99-2zv2v" event={"ID":"4822efcb-f15f-4857-b882-774b0baec527","Type":"ContainerDied","Data":"4848194a8915c507deeb8806f55974114a9e40b94698bded0fe1383a319489c9"} Sep 08 18:41:48.210383 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:48.210335 2555 scope.go:117] "RemoveContainer" containerID="442291376e5bd98cc82f1abebe1e7731665244be47d11ef881e803356c571a79" Sep 08 18:41:48.211742 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:48.211717 2555 generic.go:356] "Generic (PLEG): container finished" podID="241061eb-fa90-45a1-ae52-13730bf5c646" containerID="9cca1fbfdcea09d33163af2fc49eb250038b3e786a9861ede59906d77c5c0b19" exitCode=0 Sep 08 18:41:48.211836 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:48.211750 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-15d796-668bc7c5d-6m7d9" Sep 08 18:41:48.211836 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:48.211788 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-15d796-668bc7c5d-6m7d9" event={"ID":"241061eb-fa90-45a1-ae52-13730bf5c646","Type":"ContainerDied","Data":"9cca1fbfdcea09d33163af2fc49eb250038b3e786a9861ede59906d77c5c0b19"} Sep 08 18:41:48.211916 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:48.211834 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-15d796-668bc7c5d-6m7d9" event={"ID":"241061eb-fa90-45a1-ae52-13730bf5c646","Type":"ContainerDied","Data":"64ef50e258e298af3150d4dc4ebdf576ec080883a4c347dd8b975ffab4d4b9e0"} Sep 08 18:41:48.220106 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:48.220090 2555 scope.go:117] "RemoveContainer" containerID="442291376e5bd98cc82f1abebe1e7731665244be47d11ef881e803356c571a79" Sep 08 18:41:48.220435 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:41:48.220416 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"442291376e5bd98cc82f1abebe1e7731665244be47d11ef881e803356c571a79\": container with ID starting with 442291376e5bd98cc82f1abebe1e7731665244be47d11ef881e803356c571a79 not found: ID does not exist" containerID="442291376e5bd98cc82f1abebe1e7731665244be47d11ef881e803356c571a79" Sep 08 18:41:48.220478 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:48.220443 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"442291376e5bd98cc82f1abebe1e7731665244be47d11ef881e803356c571a79"} err="failed to get container status \"442291376e5bd98cc82f1abebe1e7731665244be47d11ef881e803356c571a79\": rpc error: code = NotFound desc = could not find container \"442291376e5bd98cc82f1abebe1e7731665244be47d11ef881e803356c571a79\": container with ID starting with 442291376e5bd98cc82f1abebe1e7731665244be47d11ef881e803356c571a79 not found: ID does not exist" Sep 08 18:41:48.220478 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:48.220457 2555 scope.go:117] "RemoveContainer" containerID="9cca1fbfdcea09d33163af2fc49eb250038b3e786a9861ede59906d77c5c0b19" Sep 08 18:41:48.229452 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:48.229432 2555 scope.go:117] "RemoveContainer" containerID="9cca1fbfdcea09d33163af2fc49eb250038b3e786a9861ede59906d77c5c0b19" Sep 08 18:41:48.229802 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:41:48.229784 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cca1fbfdcea09d33163af2fc49eb250038b3e786a9861ede59906d77c5c0b19\": container with ID starting with 9cca1fbfdcea09d33163af2fc49eb250038b3e786a9861ede59906d77c5c0b19 not found: ID does not exist" containerID="9cca1fbfdcea09d33163af2fc49eb250038b3e786a9861ede59906d77c5c0b19" Sep 08 18:41:48.229873 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:48.229810 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cca1fbfdcea09d33163af2fc49eb250038b3e786a9861ede59906d77c5c0b19"} err="failed to get container status \"9cca1fbfdcea09d33163af2fc49eb250038b3e786a9861ede59906d77c5c0b19\": rpc error: code = NotFound desc = could not find container \"9cca1fbfdcea09d33163af2fc49eb250038b3e786a9861ede59906d77c5c0b19\": container with ID starting with 9cca1fbfdcea09d33163af2fc49eb250038b3e786a9861ede59906d77c5c0b19 not found: ID does not exist" Sep 08 18:41:48.247154 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:48.247125 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nhgsg\" (UniqueName: \"kubernetes.io/projected/241061eb-fa90-45a1-ae52-13730bf5c646-kube-api-access-nhgsg\") pod \"241061eb-fa90-45a1-ae52-13730bf5c646\" (UID: \"241061eb-fa90-45a1-ae52-13730bf5c646\") " Sep 08 18:41:48.247591 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:48.247202 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hsl2b\" (UniqueName: \"kubernetes.io/projected/4822efcb-f15f-4857-b882-774b0baec527-kube-api-access-hsl2b\") pod \"4822efcb-f15f-4857-b882-774b0baec527\" (UID: \"4822efcb-f15f-4857-b882-774b0baec527\") " Sep 08 18:41:48.247591 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:48.247233 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4822efcb-f15f-4857-b882-774b0baec527-plugins-config-volume\") pod \"4822efcb-f15f-4857-b882-774b0baec527\" (UID: \"4822efcb-f15f-4857-b882-774b0baec527\") " Sep 08 18:41:48.247591 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:48.247286 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/241061eb-fa90-45a1-ae52-13730bf5c646-plugins-config-volume\") pod \"241061eb-fa90-45a1-ae52-13730bf5c646\" (UID: \"241061eb-fa90-45a1-ae52-13730bf5c646\") " Sep 08 18:41:48.247813 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:48.247716 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/241061eb-fa90-45a1-ae52-13730bf5c646-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "241061eb-fa90-45a1-ae52-13730bf5c646" (UID: "241061eb-fa90-45a1-ae52-13730bf5c646"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:41:48.247813 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:48.247720 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4822efcb-f15f-4857-b882-774b0baec527-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4822efcb-f15f-4857-b882-774b0baec527" (UID: "4822efcb-f15f-4857-b882-774b0baec527"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:41:48.249488 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:48.249464 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4822efcb-f15f-4857-b882-774b0baec527-kube-api-access-hsl2b" (OuterVolumeSpecName: "kube-api-access-hsl2b") pod "4822efcb-f15f-4857-b882-774b0baec527" (UID: "4822efcb-f15f-4857-b882-774b0baec527"). InnerVolumeSpecName "kube-api-access-hsl2b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:41:48.249593 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:48.249493 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/241061eb-fa90-45a1-ae52-13730bf5c646-kube-api-access-nhgsg" (OuterVolumeSpecName: "kube-api-access-nhgsg") pod "241061eb-fa90-45a1-ae52-13730bf5c646" (UID: "241061eb-fa90-45a1-ae52-13730bf5c646"). InnerVolumeSpecName "kube-api-access-nhgsg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:41:48.336782 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:48.336757 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:48.348207 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:48.348164 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhgsg\" (UniqueName: \"kubernetes.io/projected/241061eb-fa90-45a1-ae52-13730bf5c646-kube-api-access-nhgsg\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:41:48.348207 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:48.348205 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hsl2b\" (UniqueName: \"kubernetes.io/projected/4822efcb-f15f-4857-b882-774b0baec527-kube-api-access-hsl2b\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:41:48.348392 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:48.348217 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4822efcb-f15f-4857-b882-774b0baec527-plugins-config-volume\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:41:48.348392 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:48.348229 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/241061eb-fa90-45a1-ae52-13730bf5c646-plugins-config-volume\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:41:48.526504 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:48.526469 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-15d796-5c8bd9fb99-2zv2v"] Sep 08 18:41:48.532827 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:48.532792 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-15d796-5c8bd9fb99-2zv2v"] Sep 08 18:41:48.542900 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:48.542864 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-15d796-668bc7c5d-6m7d9"] Sep 08 18:41:48.548591 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:48.548559 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-15d796-668bc7c5d-6m7d9"] Sep 08 18:41:50.325093 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:50.325054 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="241061eb-fa90-45a1-ae52-13730bf5c646" path="/var/lib/kubelet/pods/241061eb-fa90-45a1-ae52-13730bf5c646/volumes" Sep 08 18:41:50.325622 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:50.325602 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4822efcb-f15f-4857-b882-774b0baec527" path="/var/lib/kubelet/pods/4822efcb-f15f-4857-b882-774b0baec527/volumes" Sep 08 18:41:50.531700 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:50.531640 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:41:52.537587 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:52.537554 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:53.096162 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:53.096125 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:53.131231 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:53.131194 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2e25b35c-67d478989d-c96tj"] Sep 08 18:41:53.131494 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:53.131439 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-2e25b35c-67d478989d-c96tj" podUID="0a022862-8679-4dbf-b8a5-d1e4496ed6be" containerName="payload-pre-processing" containerID="cri-o://247b8cc2cef468119d49b7ec69a576288885351d3dd6b8d39c3eee7d59aabb01" gracePeriod=30 Sep 08 18:41:53.196833 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:53.196806 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:53.278228 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:53.278197 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:53.341444 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:53.341416 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:53.377504 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:53.377484 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2e25b35c-67d478989d-c96tj" Sep 08 18:41:53.394867 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:53.394840 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a022862-8679-4dbf-b8a5-d1e4496ed6be-plugins-config-volume\") pod \"0a022862-8679-4dbf-b8a5-d1e4496ed6be\" (UID: \"0a022862-8679-4dbf-b8a5-d1e4496ed6be\") " Sep 08 18:41:53.395420 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:53.394929 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5njm\" (UniqueName: \"kubernetes.io/projected/0a022862-8679-4dbf-b8a5-d1e4496ed6be-kube-api-access-h5njm\") pod \"0a022862-8679-4dbf-b8a5-d1e4496ed6be\" (UID: \"0a022862-8679-4dbf-b8a5-d1e4496ed6be\") " Sep 08 18:41:53.395420 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:53.395246 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a022862-8679-4dbf-b8a5-d1e4496ed6be-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0a022862-8679-4dbf-b8a5-d1e4496ed6be" (UID: "0a022862-8679-4dbf-b8a5-d1e4496ed6be"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:41:53.397732 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:53.397706 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a022862-8679-4dbf-b8a5-d1e4496ed6be-kube-api-access-h5njm" (OuterVolumeSpecName: "kube-api-access-h5njm") pod "0a022862-8679-4dbf-b8a5-d1e4496ed6be" (UID: "0a022862-8679-4dbf-b8a5-d1e4496ed6be"). InnerVolumeSpecName "kube-api-access-h5njm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:41:53.496407 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:53.496372 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5njm\" (UniqueName: \"kubernetes.io/projected/0a022862-8679-4dbf-b8a5-d1e4496ed6be-kube-api-access-h5njm\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:41:53.496407 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:53.496406 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a022862-8679-4dbf-b8a5-d1e4496ed6be-plugins-config-volume\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:41:53.637729 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:53.637635 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:54.251124 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:54.251089 2555 generic.go:356] "Generic (PLEG): container finished" podID="0a022862-8679-4dbf-b8a5-d1e4496ed6be" containerID="247b8cc2cef468119d49b7ec69a576288885351d3dd6b8d39c3eee7d59aabb01" exitCode=0 Sep 08 18:41:54.251124 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:54.251125 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2e25b35c-67d478989d-c96tj" event={"ID":"0a022862-8679-4dbf-b8a5-d1e4496ed6be","Type":"ContainerDied","Data":"247b8cc2cef468119d49b7ec69a576288885351d3dd6b8d39c3eee7d59aabb01"} Sep 08 18:41:54.251340 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:54.251134 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2e25b35c-67d478989d-c96tj" Sep 08 18:41:54.251340 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:54.251162 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2e25b35c-67d478989d-c96tj" event={"ID":"0a022862-8679-4dbf-b8a5-d1e4496ed6be","Type":"ContainerDied","Data":"aef43c6809e594a6a682e2f014fff03b62180d4cd6ed8c174bc158433e6dbfdb"} Sep 08 18:41:54.251340 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:54.251184 2555 scope.go:117] "RemoveContainer" containerID="247b8cc2cef468119d49b7ec69a576288885351d3dd6b8d39c3eee7d59aabb01" Sep 08 18:41:54.260337 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:54.260321 2555 scope.go:117] "RemoveContainer" containerID="247b8cc2cef468119d49b7ec69a576288885351d3dd6b8d39c3eee7d59aabb01" Sep 08 18:41:54.260603 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:41:54.260586 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"247b8cc2cef468119d49b7ec69a576288885351d3dd6b8d39c3eee7d59aabb01\": container with ID starting with 247b8cc2cef468119d49b7ec69a576288885351d3dd6b8d39c3eee7d59aabb01 not found: ID does not exist" containerID="247b8cc2cef468119d49b7ec69a576288885351d3dd6b8d39c3eee7d59aabb01" Sep 08 18:41:54.260651 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:54.260611 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"247b8cc2cef468119d49b7ec69a576288885351d3dd6b8d39c3eee7d59aabb01"} err="failed to get container status \"247b8cc2cef468119d49b7ec69a576288885351d3dd6b8d39c3eee7d59aabb01\": rpc error: code = NotFound desc = could not find container \"247b8cc2cef468119d49b7ec69a576288885351d3dd6b8d39c3eee7d59aabb01\": container with ID starting with 247b8cc2cef468119d49b7ec69a576288885351d3dd6b8d39c3eee7d59aabb01 not found: ID does not exist" Sep 08 18:41:54.271855 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:54.271833 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2e25b35c-67d478989d-c96tj"] Sep 08 18:41:54.278151 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:54.278129 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2e25b35c-67d478989d-c96tj"] Sep 08 18:41:54.324773 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:54.324752 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a022862-8679-4dbf-b8a5-d1e4496ed6be" path="/var/lib/kubelet/pods/0a022862-8679-4dbf-b8a5-d1e4496ed6be/volumes" Sep 08 18:41:54.468986 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:54.468960 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:54.569612 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:54.569584 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:54.724365 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:54.724333 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw"] Sep 08 18:41:54.724857 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:54.724576 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" podUID="ed6265b9-7057-49d9-85e6-9b08b0809029" containerName="istio-proxy" containerID="cri-o://2d8855908839af3b4d98f8070c9d8e487b78f8c978b5c177cd63b05baa84db02" gracePeriod=30 Sep 08 18:41:54.746505 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:54.746449 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:55.314470 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:55.314431 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:55.591525 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:55.591420 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:56.172853 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:56.172825 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:58.251117 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:58.251060 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:58.347127 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:58.347093 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:59.029429 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:59.029386 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:41:59.567003 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:59.566971 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:59.693283 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:59.693252 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:59.850011 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:59.849981 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj"] Sep 08 18:41:59.850239 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:59.850215 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" podUID="3ec70540-6806-4667-aed6-c4891a8ae835" containerName="istio-proxy" containerID="cri-o://b20ac0b79e49cb8520bcd854a4da63e89dab42dc8ed10e7daf53d8a35809318a" gracePeriod=30 Sep 08 18:41:59.861108 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:59.861068 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:59.966248 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:59.966226 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:41:59.986782 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:41:59.986759 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:42:00.054596 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.054563 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed6265b9-7057-49d9-85e6-9b08b0809029-istio-data\") pod \"ed6265b9-7057-49d9-85e6-9b08b0809029\" (UID: \"ed6265b9-7057-49d9-85e6-9b08b0809029\") " Sep 08 18:42:00.054799 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.054611 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed6265b9-7057-49d9-85e6-9b08b0809029-credential-socket\") pod \"ed6265b9-7057-49d9-85e6-9b08b0809029\" (UID: \"ed6265b9-7057-49d9-85e6-9b08b0809029\") " Sep 08 18:42:00.054799 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.054639 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8x244\" (UniqueName: \"kubernetes.io/projected/ed6265b9-7057-49d9-85e6-9b08b0809029-kube-api-access-8x244\") pod \"ed6265b9-7057-49d9-85e6-9b08b0809029\" (UID: \"ed6265b9-7057-49d9-85e6-9b08b0809029\") " Sep 08 18:42:00.054799 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.054710 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed6265b9-7057-49d9-85e6-9b08b0809029-workload-socket\") pod \"ed6265b9-7057-49d9-85e6-9b08b0809029\" (UID: \"ed6265b9-7057-49d9-85e6-9b08b0809029\") " Sep 08 18:42:00.054799 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.054737 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed6265b9-7057-49d9-85e6-9b08b0809029-workload-certs\") pod \"ed6265b9-7057-49d9-85e6-9b08b0809029\" (UID: \"ed6265b9-7057-49d9-85e6-9b08b0809029\") " Sep 08 18:42:00.055009 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.054869 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed6265b9-7057-49d9-85e6-9b08b0809029-istio-podinfo\") pod \"ed6265b9-7057-49d9-85e6-9b08b0809029\" (UID: \"ed6265b9-7057-49d9-85e6-9b08b0809029\") " Sep 08 18:42:00.055009 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.054914 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed6265b9-7057-49d9-85e6-9b08b0809029-istio-data" (OuterVolumeSpecName: "istio-data") pod "ed6265b9-7057-49d9-85e6-9b08b0809029" (UID: "ed6265b9-7057-49d9-85e6-9b08b0809029"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:42:00.055009 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.054941 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed6265b9-7057-49d9-85e6-9b08b0809029-istio-envoy\") pod \"ed6265b9-7057-49d9-85e6-9b08b0809029\" (UID: \"ed6265b9-7057-49d9-85e6-9b08b0809029\") " Sep 08 18:42:00.055009 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.054948 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed6265b9-7057-49d9-85e6-9b08b0809029-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ed6265b9-7057-49d9-85e6-9b08b0809029" (UID: "ed6265b9-7057-49d9-85e6-9b08b0809029"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:42:00.055009 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.055001 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed6265b9-7057-49d9-85e6-9b08b0809029-istio-token\") pod \"ed6265b9-7057-49d9-85e6-9b08b0809029\" (UID: \"ed6265b9-7057-49d9-85e6-9b08b0809029\") " Sep 08 18:42:00.055263 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.055045 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed6265b9-7057-49d9-85e6-9b08b0809029-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ed6265b9-7057-49d9-85e6-9b08b0809029" (UID: "ed6265b9-7057-49d9-85e6-9b08b0809029"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:42:00.055263 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.055058 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed6265b9-7057-49d9-85e6-9b08b0809029-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ed6265b9-7057-49d9-85e6-9b08b0809029" (UID: "ed6265b9-7057-49d9-85e6-9b08b0809029"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:42:00.055263 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.055070 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed6265b9-7057-49d9-85e6-9b08b0809029-istiod-ca-cert\") pod \"ed6265b9-7057-49d9-85e6-9b08b0809029\" (UID: \"ed6265b9-7057-49d9-85e6-9b08b0809029\") " Sep 08 18:42:00.055403 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.055321 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed6265b9-7057-49d9-85e6-9b08b0809029-istio-data\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:42:00.055403 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.055338 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed6265b9-7057-49d9-85e6-9b08b0809029-credential-socket\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:42:00.055403 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.055350 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed6265b9-7057-49d9-85e6-9b08b0809029-workload-socket\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:42:00.055403 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.055359 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed6265b9-7057-49d9-85e6-9b08b0809029-workload-certs\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:42:00.055788 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.055444 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed6265b9-7057-49d9-85e6-9b08b0809029-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ed6265b9-7057-49d9-85e6-9b08b0809029" (UID: "ed6265b9-7057-49d9-85e6-9b08b0809029"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:42:00.057217 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.057192 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed6265b9-7057-49d9-85e6-9b08b0809029-kube-api-access-8x244" (OuterVolumeSpecName: "kube-api-access-8x244") pod "ed6265b9-7057-49d9-85e6-9b08b0809029" (UID: "ed6265b9-7057-49d9-85e6-9b08b0809029"). InnerVolumeSpecName "kube-api-access-8x244". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:42:00.057217 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.057195 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed6265b9-7057-49d9-85e6-9b08b0809029-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ed6265b9-7057-49d9-85e6-9b08b0809029" (UID: "ed6265b9-7057-49d9-85e6-9b08b0809029"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:42:00.057384 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.057219 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ed6265b9-7057-49d9-85e6-9b08b0809029-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ed6265b9-7057-49d9-85e6-9b08b0809029" (UID: "ed6265b9-7057-49d9-85e6-9b08b0809029"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 08 18:42:00.057384 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.057328 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed6265b9-7057-49d9-85e6-9b08b0809029-istio-token" (OuterVolumeSpecName: "istio-token") pod "ed6265b9-7057-49d9-85e6-9b08b0809029" (UID: "ed6265b9-7057-49d9-85e6-9b08b0809029"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:42:00.150344 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.150232 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:00.156466 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.156439 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8x244\" (UniqueName: \"kubernetes.io/projected/ed6265b9-7057-49d9-85e6-9b08b0809029-kube-api-access-8x244\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:42:00.156466 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.156466 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed6265b9-7057-49d9-85e6-9b08b0809029-istio-podinfo\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:42:00.156629 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.156475 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed6265b9-7057-49d9-85e6-9b08b0809029-istio-envoy\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:42:00.156629 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.156483 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed6265b9-7057-49d9-85e6-9b08b0809029-istio-token\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:42:00.156629 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.156491 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed6265b9-7057-49d9-85e6-9b08b0809029-istiod-ca-cert\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:42:00.251953 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.251920 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:00.289372 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.289342 2555 generic.go:356] "Generic (PLEG): container finished" podID="ed6265b9-7057-49d9-85e6-9b08b0809029" containerID="2d8855908839af3b4d98f8070c9d8e487b78f8c978b5c177cd63b05baa84db02" exitCode=0 Sep 08 18:42:00.289518 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.289389 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" Sep 08 18:42:00.289518 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.289432 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" event={"ID":"ed6265b9-7057-49d9-85e6-9b08b0809029","Type":"ContainerDied","Data":"2d8855908839af3b4d98f8070c9d8e487b78f8c978b5c177cd63b05baa84db02"} Sep 08 18:42:00.289518 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.289496 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw" event={"ID":"ed6265b9-7057-49d9-85e6-9b08b0809029","Type":"ContainerDied","Data":"91af66a560460b08f6c99988a5fdd00e12301d04d94323572d354b2f5acb24e0"} Sep 08 18:42:00.289657 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.289536 2555 scope.go:117] "RemoveContainer" containerID="2d8855908839af3b4d98f8070c9d8e487b78f8c978b5c177cd63b05baa84db02" Sep 08 18:42:00.302631 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.302473 2555 scope.go:117] "RemoveContainer" containerID="f8692b6822a42ac57622d7f993cc0fe6293ade303beda66bb8ed2dacf82d713f" Sep 08 18:42:00.311649 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.311630 2555 scope.go:117] "RemoveContainer" containerID="2d8855908839af3b4d98f8070c9d8e487b78f8c978b5c177cd63b05baa84db02" Sep 08 18:42:00.311980 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:42:00.311961 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d8855908839af3b4d98f8070c9d8e487b78f8c978b5c177cd63b05baa84db02\": container with ID starting with 2d8855908839af3b4d98f8070c9d8e487b78f8c978b5c177cd63b05baa84db02 not found: ID does not exist" containerID="2d8855908839af3b4d98f8070c9d8e487b78f8c978b5c177cd63b05baa84db02" Sep 08 18:42:00.312036 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.311989 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d8855908839af3b4d98f8070c9d8e487b78f8c978b5c177cd63b05baa84db02"} err="failed to get container status \"2d8855908839af3b4d98f8070c9d8e487b78f8c978b5c177cd63b05baa84db02\": rpc error: code = NotFound desc = could not find container \"2d8855908839af3b4d98f8070c9d8e487b78f8c978b5c177cd63b05baa84db02\": container with ID starting with 2d8855908839af3b4d98f8070c9d8e487b78f8c978b5c177cd63b05baa84db02 not found: ID does not exist" Sep 08 18:42:00.312036 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.312007 2555 scope.go:117] "RemoveContainer" containerID="f8692b6822a42ac57622d7f993cc0fe6293ade303beda66bb8ed2dacf82d713f" Sep 08 18:42:00.312270 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:42:00.312253 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8692b6822a42ac57622d7f993cc0fe6293ade303beda66bb8ed2dacf82d713f\": container with ID starting with f8692b6822a42ac57622d7f993cc0fe6293ade303beda66bb8ed2dacf82d713f not found: ID does not exist" containerID="f8692b6822a42ac57622d7f993cc0fe6293ade303beda66bb8ed2dacf82d713f" Sep 08 18:42:00.312311 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.312277 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8692b6822a42ac57622d7f993cc0fe6293ade303beda66bb8ed2dacf82d713f"} err="failed to get container status \"f8692b6822a42ac57622d7f993cc0fe6293ade303beda66bb8ed2dacf82d713f\": rpc error: code = NotFound desc = could not find container \"f8692b6822a42ac57622d7f993cc0fe6293ade303beda66bb8ed2dacf82d713f\": container with ID starting with f8692b6822a42ac57622d7f993cc0fe6293ade303beda66bb8ed2dacf82d713f not found: ID does not exist" Sep 08 18:42:00.317841 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.317815 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw"] Sep 08 18:42:00.325556 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.325521 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-15d796-openshift-default-756bcdf87b-kk4xw"] Sep 08 18:42:00.352332 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.352307 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:00.453973 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:00.453878 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:01.661668 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:01.661635 2555 patch_prober.go:28] interesting pod/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 08 18:42:01.662072 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:01.661741 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" podUID="3ec70540-6806-4667-aed6-c4891a8ae835" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 08 18:42:01.800728 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:01.800678 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:02.324513 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:02.324479 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed6265b9-7057-49d9-85e6-9b08b0809029" path="/var/lib/kubelet/pods/ed6265b9-7057-49d9-85e6-9b08b0809029/volumes" Sep 08 18:42:05.103512 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.103487 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:42:05.200410 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.200321 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ec70540-6806-4667-aed6-c4891a8ae835-istio-envoy\") pod \"3ec70540-6806-4667-aed6-c4891a8ae835\" (UID: \"3ec70540-6806-4667-aed6-c4891a8ae835\") " Sep 08 18:42:05.200410 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.200360 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ec70540-6806-4667-aed6-c4891a8ae835-istiod-ca-cert\") pod \"3ec70540-6806-4667-aed6-c4891a8ae835\" (UID: \"3ec70540-6806-4667-aed6-c4891a8ae835\") " Sep 08 18:42:05.200410 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.200387 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wd278\" (UniqueName: \"kubernetes.io/projected/3ec70540-6806-4667-aed6-c4891a8ae835-kube-api-access-wd278\") pod \"3ec70540-6806-4667-aed6-c4891a8ae835\" (UID: \"3ec70540-6806-4667-aed6-c4891a8ae835\") " Sep 08 18:42:05.200726 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.200424 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ec70540-6806-4667-aed6-c4891a8ae835-istio-data\") pod \"3ec70540-6806-4667-aed6-c4891a8ae835\" (UID: \"3ec70540-6806-4667-aed6-c4891a8ae835\") " Sep 08 18:42:05.200726 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.200457 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ec70540-6806-4667-aed6-c4891a8ae835-credential-socket\") pod \"3ec70540-6806-4667-aed6-c4891a8ae835\" (UID: \"3ec70540-6806-4667-aed6-c4891a8ae835\") " Sep 08 18:42:05.200726 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.200486 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ec70540-6806-4667-aed6-c4891a8ae835-workload-certs\") pod \"3ec70540-6806-4667-aed6-c4891a8ae835\" (UID: \"3ec70540-6806-4667-aed6-c4891a8ae835\") " Sep 08 18:42:05.200726 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.200516 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ec70540-6806-4667-aed6-c4891a8ae835-workload-socket\") pod \"3ec70540-6806-4667-aed6-c4891a8ae835\" (UID: \"3ec70540-6806-4667-aed6-c4891a8ae835\") " Sep 08 18:42:05.200726 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.200537 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ec70540-6806-4667-aed6-c4891a8ae835-istio-token\") pod \"3ec70540-6806-4667-aed6-c4891a8ae835\" (UID: \"3ec70540-6806-4667-aed6-c4891a8ae835\") " Sep 08 18:42:05.200726 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.200561 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ec70540-6806-4667-aed6-c4891a8ae835-istio-podinfo\") pod \"3ec70540-6806-4667-aed6-c4891a8ae835\" (UID: \"3ec70540-6806-4667-aed6-c4891a8ae835\") " Sep 08 18:42:05.201045 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.200822 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ec70540-6806-4667-aed6-c4891a8ae835-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3ec70540-6806-4667-aed6-c4891a8ae835" (UID: "3ec70540-6806-4667-aed6-c4891a8ae835"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:42:05.201045 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.200842 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ec70540-6806-4667-aed6-c4891a8ae835-istio-data" (OuterVolumeSpecName: "istio-data") pod "3ec70540-6806-4667-aed6-c4891a8ae835" (UID: "3ec70540-6806-4667-aed6-c4891a8ae835"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:42:05.201045 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.200892 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ec70540-6806-4667-aed6-c4891a8ae835-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3ec70540-6806-4667-aed6-c4891a8ae835" (UID: "3ec70540-6806-4667-aed6-c4891a8ae835"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:42:05.201045 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.200895 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ec70540-6806-4667-aed6-c4891a8ae835-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3ec70540-6806-4667-aed6-c4891a8ae835" (UID: "3ec70540-6806-4667-aed6-c4891a8ae835"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:42:05.201045 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.201001 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ec70540-6806-4667-aed6-c4891a8ae835-istio-data\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:42:05.201045 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.201019 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ec70540-6806-4667-aed6-c4891a8ae835-credential-socket\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:42:05.201045 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.201032 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ec70540-6806-4667-aed6-c4891a8ae835-workload-socket\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:42:05.201045 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.201046 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ec70540-6806-4667-aed6-c4891a8ae835-istiod-ca-cert\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:42:05.201431 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.201072 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ec70540-6806-4667-aed6-c4891a8ae835-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3ec70540-6806-4667-aed6-c4891a8ae835" (UID: "3ec70540-6806-4667-aed6-c4891a8ae835"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:42:05.202810 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.202788 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ec70540-6806-4667-aed6-c4891a8ae835-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3ec70540-6806-4667-aed6-c4891a8ae835" (UID: "3ec70540-6806-4667-aed6-c4891a8ae835"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:42:05.203077 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.203055 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ec70540-6806-4667-aed6-c4891a8ae835-istio-token" (OuterVolumeSpecName: "istio-token") pod "3ec70540-6806-4667-aed6-c4891a8ae835" (UID: "3ec70540-6806-4667-aed6-c4891a8ae835"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:42:05.203133 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.203055 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ec70540-6806-4667-aed6-c4891a8ae835-kube-api-access-wd278" (OuterVolumeSpecName: "kube-api-access-wd278") pod "3ec70540-6806-4667-aed6-c4891a8ae835" (UID: "3ec70540-6806-4667-aed6-c4891a8ae835"). InnerVolumeSpecName "kube-api-access-wd278". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:42:05.203133 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.203058 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3ec70540-6806-4667-aed6-c4891a8ae835-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3ec70540-6806-4667-aed6-c4891a8ae835" (UID: "3ec70540-6806-4667-aed6-c4891a8ae835"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 08 18:42:05.302361 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.302322 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ec70540-6806-4667-aed6-c4891a8ae835-istio-envoy\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:42:05.302361 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.302355 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wd278\" (UniqueName: \"kubernetes.io/projected/3ec70540-6806-4667-aed6-c4891a8ae835-kube-api-access-wd278\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:42:05.302361 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.302367 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ec70540-6806-4667-aed6-c4891a8ae835-workload-certs\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:42:05.302611 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.302377 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ec70540-6806-4667-aed6-c4891a8ae835-istio-token\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:42:05.302611 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.302384 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ec70540-6806-4667-aed6-c4891a8ae835-istio-podinfo\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:42:05.321763 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.321730 2555 generic.go:356] "Generic (PLEG): container finished" podID="3ec70540-6806-4667-aed6-c4891a8ae835" containerID="b20ac0b79e49cb8520bcd854a4da63e89dab42dc8ed10e7daf53d8a35809318a" exitCode=0 Sep 08 18:42:05.321908 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.321787 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" Sep 08 18:42:05.321908 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.321822 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" event={"ID":"3ec70540-6806-4667-aed6-c4891a8ae835","Type":"ContainerDied","Data":"b20ac0b79e49cb8520bcd854a4da63e89dab42dc8ed10e7daf53d8a35809318a"} Sep 08 18:42:05.321908 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.321871 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj" event={"ID":"3ec70540-6806-4667-aed6-c4891a8ae835","Type":"ContainerDied","Data":"ce7afc125cd4da3c6a638198de8c11800b14906819a19ed07a8d2c54a27f1b58"} Sep 08 18:42:05.321908 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.321892 2555 scope.go:117] "RemoveContainer" containerID="b20ac0b79e49cb8520bcd854a4da63e89dab42dc8ed10e7daf53d8a35809318a" Sep 08 18:42:05.331931 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.331910 2555 scope.go:117] "RemoveContainer" containerID="e069074cec3ece1c7f9e3a542fc2dfc6e2218b3cea720d46b91f211fb8b4e583" Sep 08 18:42:05.340753 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.340735 2555 scope.go:117] "RemoveContainer" containerID="b20ac0b79e49cb8520bcd854a4da63e89dab42dc8ed10e7daf53d8a35809318a" Sep 08 18:42:05.341043 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:42:05.341024 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b20ac0b79e49cb8520bcd854a4da63e89dab42dc8ed10e7daf53d8a35809318a\": container with ID starting with b20ac0b79e49cb8520bcd854a4da63e89dab42dc8ed10e7daf53d8a35809318a not found: ID does not exist" containerID="b20ac0b79e49cb8520bcd854a4da63e89dab42dc8ed10e7daf53d8a35809318a" Sep 08 18:42:05.341115 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.341054 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b20ac0b79e49cb8520bcd854a4da63e89dab42dc8ed10e7daf53d8a35809318a"} err="failed to get container status \"b20ac0b79e49cb8520bcd854a4da63e89dab42dc8ed10e7daf53d8a35809318a\": rpc error: code = NotFound desc = could not find container \"b20ac0b79e49cb8520bcd854a4da63e89dab42dc8ed10e7daf53d8a35809318a\": container with ID starting with b20ac0b79e49cb8520bcd854a4da63e89dab42dc8ed10e7daf53d8a35809318a not found: ID does not exist" Sep 08 18:42:05.341115 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.341071 2555 scope.go:117] "RemoveContainer" containerID="e069074cec3ece1c7f9e3a542fc2dfc6e2218b3cea720d46b91f211fb8b4e583" Sep 08 18:42:05.341374 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:42:05.341355 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e069074cec3ece1c7f9e3a542fc2dfc6e2218b3cea720d46b91f211fb8b4e583\": container with ID starting with e069074cec3ece1c7f9e3a542fc2dfc6e2218b3cea720d46b91f211fb8b4e583 not found: ID does not exist" containerID="e069074cec3ece1c7f9e3a542fc2dfc6e2218b3cea720d46b91f211fb8b4e583" Sep 08 18:42:05.341420 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.341380 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e069074cec3ece1c7f9e3a542fc2dfc6e2218b3cea720d46b91f211fb8b4e583"} err="failed to get container status \"e069074cec3ece1c7f9e3a542fc2dfc6e2218b3cea720d46b91f211fb8b4e583\": rpc error: code = NotFound desc = could not find container \"e069074cec3ece1c7f9e3a542fc2dfc6e2218b3cea720d46b91f211fb8b4e583\": container with ID starting with e069074cec3ece1c7f9e3a542fc2dfc6e2218b3cea720d46b91f211fb8b4e583 not found: ID does not exist" Sep 08 18:42:05.343133 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.343115 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj"] Sep 08 18:42:05.346739 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:05.346719 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-2e25b35c-openshift-default-565c6c4bd4-mfsjj"] Sep 08 18:42:06.324947 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:06.324917 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ec70540-6806-4667-aed6-c4891a8ae835" path="/var/lib/kubelet/pods/3ec70540-6806-4667-aed6-c4891a8ae835/volumes" Sep 08 18:42:06.737514 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:06.737412 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:06.837919 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:06.837888 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:06.936447 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:06.936414 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:07.073184 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:07.073154 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:07.534332 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:07.534243 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:42:13.148033 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:13.147998 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:13.251218 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:13.251164 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:14.056110 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:14.056033 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:14.378894 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:14.378799 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:15.029898 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:15.029868 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:17.365326 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:17.365281 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:17.471138 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:17.471108 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:18.668155 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:18.668123 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:18.778035 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:18.778004 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:18.916515 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:18.916485 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:19.201843 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.201810 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:19.300582 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.300551 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:19.320663 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.320630 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-3b04d6-5f657ddf9f-6vk9h"] Sep 08 18:42:19.321019 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.321006 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed6265b9-7057-49d9-85e6-9b08b0809029" containerName="istio-proxy" Sep 08 18:42:19.321063 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.321020 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed6265b9-7057-49d9-85e6-9b08b0809029" containerName="istio-proxy" Sep 08 18:42:19.321063 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.321030 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="241061eb-fa90-45a1-ae52-13730bf5c646" containerName="payload-pre-processing" Sep 08 18:42:19.321063 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.321035 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="241061eb-fa90-45a1-ae52-13730bf5c646" containerName="payload-pre-processing" Sep 08 18:42:19.321063 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.321049 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ec70540-6806-4667-aed6-c4891a8ae835" containerName="istio-proxy" Sep 08 18:42:19.321063 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.321054 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ec70540-6806-4667-aed6-c4891a8ae835" containerName="istio-proxy" Sep 08 18:42:19.321063 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.321062 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4822efcb-f15f-4857-b882-774b0baec527" containerName="payload-processing" Sep 08 18:42:19.321246 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.321067 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="4822efcb-f15f-4857-b882-774b0baec527" containerName="payload-processing" Sep 08 18:42:19.321246 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.321074 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ec70540-6806-4667-aed6-c4891a8ae835" containerName="istio-proxy" Sep 08 18:42:19.321246 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.321080 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ec70540-6806-4667-aed6-c4891a8ae835" containerName="istio-proxy" Sep 08 18:42:19.321246 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.321092 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a022862-8679-4dbf-b8a5-d1e4496ed6be" containerName="payload-pre-processing" Sep 08 18:42:19.321246 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.321096 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a022862-8679-4dbf-b8a5-d1e4496ed6be" containerName="payload-pre-processing" Sep 08 18:42:19.321246 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.321149 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="4822efcb-f15f-4857-b882-774b0baec527" containerName="payload-processing" Sep 08 18:42:19.321246 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.321156 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ec70540-6806-4667-aed6-c4891a8ae835" containerName="istio-proxy" Sep 08 18:42:19.321246 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.321161 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a022862-8679-4dbf-b8a5-d1e4496ed6be" containerName="payload-pre-processing" Sep 08 18:42:19.321246 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.321168 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ec70540-6806-4667-aed6-c4891a8ae835" containerName="istio-proxy" Sep 08 18:42:19.321246 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.321175 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="241061eb-fa90-45a1-ae52-13730bf5c646" containerName="payload-pre-processing" Sep 08 18:42:19.321246 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.321181 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed6265b9-7057-49d9-85e6-9b08b0809029" containerName="istio-proxy" Sep 08 18:42:19.321246 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.321187 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed6265b9-7057-49d9-85e6-9b08b0809029" containerName="istio-proxy" Sep 08 18:42:19.325762 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.325743 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3b04d6-5f657ddf9f-6vk9h" Sep 08 18:42:19.328220 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.328199 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-3b04d6\"" Sep 08 18:42:19.328415 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.328397 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-3b04d6-dockercfg-2nsck\"" Sep 08 18:42:19.338610 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.338584 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-3b04d6-5f657ddf9f-6vk9h"] Sep 08 18:42:19.376911 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.376881 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b04d6-86d98c5cf8-jx6hr"] Sep 08 18:42:19.377279 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.377268 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed6265b9-7057-49d9-85e6-9b08b0809029" containerName="istio-proxy" Sep 08 18:42:19.377327 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.377281 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed6265b9-7057-49d9-85e6-9b08b0809029" containerName="istio-proxy" Sep 08 18:42:19.380392 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.380376 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b04d6-86d98c5cf8-jx6hr" Sep 08 18:42:19.398453 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.398426 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b04d6-86d98c5cf8-jx6hr"] Sep 08 18:42:19.435652 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.435619 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7492083a-66f1-4624-9662-86ba98931e80-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-3b04d6-5f657ddf9f-6vk9h\" (UID: \"7492083a-66f1-4624-9662-86ba98931e80\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3b04d6-5f657ddf9f-6vk9h" Sep 08 18:42:19.435858 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.435837 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vgqs\" (UniqueName: \"kubernetes.io/projected/7492083a-66f1-4624-9662-86ba98931e80-kube-api-access-4vgqs\") pod \"payload-processing-e2e-shared-a-w2-3b04d6-5f657ddf9f-6vk9h\" (UID: \"7492083a-66f1-4624-9662-86ba98931e80\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3b04d6-5f657ddf9f-6vk9h" Sep 08 18:42:19.537304 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.537274 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8sv6\" (UniqueName: \"kubernetes.io/projected/a5ee600c-bcc2-476c-beb8-cf617f8008fb-kube-api-access-b8sv6\") pod \"payload-pre-processing-e2e-shared-a-w2-3b04d6-86d98c5cf8-jx6hr\" (UID: \"a5ee600c-bcc2-476c-beb8-cf617f8008fb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b04d6-86d98c5cf8-jx6hr" Sep 08 18:42:19.537458 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.537322 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vgqs\" (UniqueName: \"kubernetes.io/projected/7492083a-66f1-4624-9662-86ba98931e80-kube-api-access-4vgqs\") pod \"payload-processing-e2e-shared-a-w2-3b04d6-5f657ddf9f-6vk9h\" (UID: \"7492083a-66f1-4624-9662-86ba98931e80\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3b04d6-5f657ddf9f-6vk9h" Sep 08 18:42:19.537458 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.537416 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7492083a-66f1-4624-9662-86ba98931e80-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-3b04d6-5f657ddf9f-6vk9h\" (UID: \"7492083a-66f1-4624-9662-86ba98931e80\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3b04d6-5f657ddf9f-6vk9h" Sep 08 18:42:19.537537 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.537456 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5ee600c-bcc2-476c-beb8-cf617f8008fb-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-3b04d6-86d98c5cf8-jx6hr\" (UID: \"a5ee600c-bcc2-476c-beb8-cf617f8008fb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b04d6-86d98c5cf8-jx6hr" Sep 08 18:42:19.538200 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.538180 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7492083a-66f1-4624-9662-86ba98931e80-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-3b04d6-5f657ddf9f-6vk9h\" (UID: \"7492083a-66f1-4624-9662-86ba98931e80\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3b04d6-5f657ddf9f-6vk9h" Sep 08 18:42:19.546225 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.546200 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vgqs\" (UniqueName: \"kubernetes.io/projected/7492083a-66f1-4624-9662-86ba98931e80-kube-api-access-4vgqs\") pod \"payload-processing-e2e-shared-a-w2-3b04d6-5f657ddf9f-6vk9h\" (UID: \"7492083a-66f1-4624-9662-86ba98931e80\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3b04d6-5f657ddf9f-6vk9h" Sep 08 18:42:19.635985 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.635947 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3b04d6-5f657ddf9f-6vk9h" Sep 08 18:42:19.638360 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.638329 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8sv6\" (UniqueName: \"kubernetes.io/projected/a5ee600c-bcc2-476c-beb8-cf617f8008fb-kube-api-access-b8sv6\") pod \"payload-pre-processing-e2e-shared-a-w2-3b04d6-86d98c5cf8-jx6hr\" (UID: \"a5ee600c-bcc2-476c-beb8-cf617f8008fb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b04d6-86d98c5cf8-jx6hr" Sep 08 18:42:19.638488 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.638398 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5ee600c-bcc2-476c-beb8-cf617f8008fb-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-3b04d6-86d98c5cf8-jx6hr\" (UID: \"a5ee600c-bcc2-476c-beb8-cf617f8008fb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b04d6-86d98c5cf8-jx6hr" Sep 08 18:42:19.639298 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.639025 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5ee600c-bcc2-476c-beb8-cf617f8008fb-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-3b04d6-86d98c5cf8-jx6hr\" (UID: \"a5ee600c-bcc2-476c-beb8-cf617f8008fb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b04d6-86d98c5cf8-jx6hr" Sep 08 18:42:19.647360 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.647328 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8sv6\" (UniqueName: \"kubernetes.io/projected/a5ee600c-bcc2-476c-beb8-cf617f8008fb-kube-api-access-b8sv6\") pod \"payload-pre-processing-e2e-shared-a-w2-3b04d6-86d98c5cf8-jx6hr\" (UID: \"a5ee600c-bcc2-476c-beb8-cf617f8008fb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b04d6-86d98c5cf8-jx6hr" Sep 08 18:42:19.695165 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.695120 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b04d6-86d98c5cf8-jx6hr" Sep 08 18:42:19.921443 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.921417 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-3b04d6-5f657ddf9f-6vk9h"] Sep 08 18:42:19.922513 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:42:19.922489 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7492083a_66f1_4624_9662_86ba98931e80.slice/crio-4a5e9761fee57754a39a2c297db591165eb70cf1d79ec93adff52e08e9ab3a36 WatchSource:0}: Error finding container 4a5e9761fee57754a39a2c297db591165eb70cf1d79ec93adff52e08e9ab3a36: Status 404 returned error can't find the container with id 4a5e9761fee57754a39a2c297db591165eb70cf1d79ec93adff52e08e9ab3a36 Sep 08 18:42:19.982321 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:19.982287 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b04d6-86d98c5cf8-jx6hr"] Sep 08 18:42:19.983820 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:42:19.983785 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5ee600c_bcc2_476c_beb8_cf617f8008fb.slice/crio-880388f148ee2be07d7c7a3322825103bac080dace76e844299f996e7d7db65f WatchSource:0}: Error finding container 880388f148ee2be07d7c7a3322825103bac080dace76e844299f996e7d7db65f: Status 404 returned error can't find the container with id 880388f148ee2be07d7c7a3322825103bac080dace76e844299f996e7d7db65f Sep 08 18:42:20.410999 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:20.410964 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3b04d6-5f657ddf9f-6vk9h" event={"ID":"7492083a-66f1-4624-9662-86ba98931e80","Type":"ContainerStarted","Data":"98e367d8f6e1fcbde685bdee8c10eb8c7745df84974cf44284586bf68f8b353a"} Sep 08 18:42:20.411220 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:20.411027 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3b04d6-5f657ddf9f-6vk9h" event={"ID":"7492083a-66f1-4624-9662-86ba98931e80","Type":"ContainerStarted","Data":"4a5e9761fee57754a39a2c297db591165eb70cf1d79ec93adff52e08e9ab3a36"} Sep 08 18:42:20.411220 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:20.411055 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3b04d6-5f657ddf9f-6vk9h" Sep 08 18:42:20.412649 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:20.412626 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b04d6-86d98c5cf8-jx6hr" event={"ID":"a5ee600c-bcc2-476c-beb8-cf617f8008fb","Type":"ContainerStarted","Data":"fea38bc5d1ce3f0ee2a365a0ff2d5dfef567076ea65170bb35843536242a9373"} Sep 08 18:42:20.412754 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:20.412665 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b04d6-86d98c5cf8-jx6hr" event={"ID":"a5ee600c-bcc2-476c-beb8-cf617f8008fb","Type":"ContainerStarted","Data":"880388f148ee2be07d7c7a3322825103bac080dace76e844299f996e7d7db65f"} Sep 08 18:42:20.412754 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:20.412713 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b04d6-86d98c5cf8-jx6hr" Sep 08 18:42:20.429996 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:20.429947 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3b04d6-5f657ddf9f-6vk9h" podStartSLOduration=1.429933332 podStartE2EDuration="1.429933332s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:42:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:42:20.427411566 +0000 UTC m=+1122.633396967" watchObservedRunningTime="2026-09-08 18:42:20.429933332 +0000 UTC m=+1122.635918747" Sep 08 18:42:20.446967 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:20.446915 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b04d6-86d98c5cf8-jx6hr" podStartSLOduration=1.446894909 podStartE2EDuration="1.446894909s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:42:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:42:20.443589847 +0000 UTC m=+1122.649575245" watchObservedRunningTime="2026-09-08 18:42:20.446894909 +0000 UTC m=+1122.652880311" Sep 08 18:42:20.876962 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:20.876929 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:22.838351 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:22.838319 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:42:23.233242 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:23.233158 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:23.482292 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:23.482260 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:23.582455 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:23.582429 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:23.818808 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:23.818780 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt"] Sep 08 18:42:23.822569 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:23.822555 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:42:23.826535 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:23.826512 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-6255f2-openshift-default-dockercfg-bttdx\"" Sep 08 18:42:23.826830 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:23.826815 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 08 18:42:23.840637 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:23.838858 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt"] Sep 08 18:42:23.857669 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:23.857629 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:23.980851 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:23.980818 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv2bw\" (UniqueName: \"kubernetes.io/projected/8ce8b8f9-b864-4072-bb13-92ad53c94917-kube-api-access-hv2bw\") pod \"e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt\" (UID: \"8ce8b8f9-b864-4072-bb13-92ad53c94917\") " pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:42:23.980851 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:23.980858 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ce8b8f9-b864-4072-bb13-92ad53c94917-credential-socket\") pod \"e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt\" (UID: \"8ce8b8f9-b864-4072-bb13-92ad53c94917\") " pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:42:23.981103 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:23.980879 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ce8b8f9-b864-4072-bb13-92ad53c94917-istio-data\") pod \"e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt\" (UID: \"8ce8b8f9-b864-4072-bb13-92ad53c94917\") " pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:42:23.981103 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:23.980939 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ce8b8f9-b864-4072-bb13-92ad53c94917-workload-socket\") pod \"e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt\" (UID: \"8ce8b8f9-b864-4072-bb13-92ad53c94917\") " pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:42:23.981103 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:23.980971 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ce8b8f9-b864-4072-bb13-92ad53c94917-istio-podinfo\") pod \"e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt\" (UID: \"8ce8b8f9-b864-4072-bb13-92ad53c94917\") " pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:42:23.981103 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:23.980985 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ce8b8f9-b864-4072-bb13-92ad53c94917-workload-certs\") pod \"e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt\" (UID: \"8ce8b8f9-b864-4072-bb13-92ad53c94917\") " pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:42:23.981103 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:23.981043 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ce8b8f9-b864-4072-bb13-92ad53c94917-istio-envoy\") pod \"e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt\" (UID: \"8ce8b8f9-b864-4072-bb13-92ad53c94917\") " pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:42:23.981285 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:23.981133 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ce8b8f9-b864-4072-bb13-92ad53c94917-istiod-ca-cert\") pod \"e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt\" (UID: \"8ce8b8f9-b864-4072-bb13-92ad53c94917\") " pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:42:23.981285 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:23.981157 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ce8b8f9-b864-4072-bb13-92ad53c94917-istio-token\") pod \"e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt\" (UID: \"8ce8b8f9-b864-4072-bb13-92ad53c94917\") " pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:42:24.082087 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:24.082050 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ce8b8f9-b864-4072-bb13-92ad53c94917-istio-envoy\") pod \"e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt\" (UID: \"8ce8b8f9-b864-4072-bb13-92ad53c94917\") " pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:42:24.082298 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:24.082106 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ce8b8f9-b864-4072-bb13-92ad53c94917-istiod-ca-cert\") pod \"e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt\" (UID: \"8ce8b8f9-b864-4072-bb13-92ad53c94917\") " pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:42:24.082298 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:24.082123 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ce8b8f9-b864-4072-bb13-92ad53c94917-istio-token\") pod \"e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt\" (UID: \"8ce8b8f9-b864-4072-bb13-92ad53c94917\") " pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:42:24.082298 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:24.082162 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv2bw\" (UniqueName: \"kubernetes.io/projected/8ce8b8f9-b864-4072-bb13-92ad53c94917-kube-api-access-hv2bw\") pod \"e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt\" (UID: \"8ce8b8f9-b864-4072-bb13-92ad53c94917\") " pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:42:24.082298 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:24.082187 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ce8b8f9-b864-4072-bb13-92ad53c94917-credential-socket\") pod \"e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt\" (UID: \"8ce8b8f9-b864-4072-bb13-92ad53c94917\") " pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:42:24.082298 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:24.082206 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ce8b8f9-b864-4072-bb13-92ad53c94917-istio-data\") pod \"e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt\" (UID: \"8ce8b8f9-b864-4072-bb13-92ad53c94917\") " pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:42:24.082574 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:24.082363 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ce8b8f9-b864-4072-bb13-92ad53c94917-workload-socket\") pod \"e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt\" (UID: \"8ce8b8f9-b864-4072-bb13-92ad53c94917\") " pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:42:24.082574 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:24.082438 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ce8b8f9-b864-4072-bb13-92ad53c94917-istio-podinfo\") pod \"e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt\" (UID: \"8ce8b8f9-b864-4072-bb13-92ad53c94917\") " pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:42:24.082574 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:24.082462 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ce8b8f9-b864-4072-bb13-92ad53c94917-workload-certs\") pod \"e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt\" (UID: \"8ce8b8f9-b864-4072-bb13-92ad53c94917\") " pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:42:24.082752 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:24.082612 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ce8b8f9-b864-4072-bb13-92ad53c94917-credential-socket\") pod \"e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt\" (UID: \"8ce8b8f9-b864-4072-bb13-92ad53c94917\") " pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:42:24.082752 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:24.082629 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ce8b8f9-b864-4072-bb13-92ad53c94917-istio-data\") pod \"e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt\" (UID: \"8ce8b8f9-b864-4072-bb13-92ad53c94917\") " pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:42:24.083018 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:24.082815 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ce8b8f9-b864-4072-bb13-92ad53c94917-workload-certs\") pod \"e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt\" (UID: \"8ce8b8f9-b864-4072-bb13-92ad53c94917\") " pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:42:24.083018 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:24.082851 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ce8b8f9-b864-4072-bb13-92ad53c94917-workload-socket\") pod \"e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt\" (UID: \"8ce8b8f9-b864-4072-bb13-92ad53c94917\") " pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:42:24.083018 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:24.082941 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ce8b8f9-b864-4072-bb13-92ad53c94917-istiod-ca-cert\") pod \"e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt\" (UID: \"8ce8b8f9-b864-4072-bb13-92ad53c94917\") " pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:42:24.084487 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:24.084466 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ce8b8f9-b864-4072-bb13-92ad53c94917-istio-envoy\") pod \"e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt\" (UID: \"8ce8b8f9-b864-4072-bb13-92ad53c94917\") " pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:42:24.084883 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:24.084862 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ce8b8f9-b864-4072-bb13-92ad53c94917-istio-podinfo\") pod \"e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt\" (UID: \"8ce8b8f9-b864-4072-bb13-92ad53c94917\") " pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:42:24.089966 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:24.089937 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ce8b8f9-b864-4072-bb13-92ad53c94917-istio-token\") pod \"e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt\" (UID: \"8ce8b8f9-b864-4072-bb13-92ad53c94917\") " pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:42:24.091068 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:24.091025 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv2bw\" (UniqueName: \"kubernetes.io/projected/8ce8b8f9-b864-4072-bb13-92ad53c94917-kube-api-access-hv2bw\") pod \"e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt\" (UID: \"8ce8b8f9-b864-4072-bb13-92ad53c94917\") " pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:42:24.134064 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:24.134035 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:42:24.414165 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:24.414140 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt"] Sep 08 18:42:24.416565 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:42:24.416535 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ce8b8f9_b864_4072_bb13_92ad53c94917.slice/crio-31c9cbe142de28541037e4e4b0d35a1bbd37f725701a3fa693b36cccd2a984fb WatchSource:0}: Error finding container 31c9cbe142de28541037e4e4b0d35a1bbd37f725701a3fa693b36cccd2a984fb: Status 404 returned error can't find the container with id 31c9cbe142de28541037e4e4b0d35a1bbd37f725701a3fa693b36cccd2a984fb Sep 08 18:42:24.419302 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:24.419271 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 18:42:24.419386 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:24.419342 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 18:42:24.419386 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:24.419369 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 18:42:24.436062 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:24.436036 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" event={"ID":"8ce8b8f9-b864-4072-bb13-92ad53c94917","Type":"ContainerStarted","Data":"31c9cbe142de28541037e4e4b0d35a1bbd37f725701a3fa693b36cccd2a984fb"} Sep 08 18:42:24.466140 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:24.466112 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:25.444341 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:25.444303 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" event={"ID":"8ce8b8f9-b864-4072-bb13-92ad53c94917","Type":"ContainerStarted","Data":"56aeee8f913e923028600b7de31543839079e013ec480ddcabdd338d9cf23353"} Sep 08 18:42:25.483844 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:25.483792 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" podStartSLOduration=2.4837759139999998 podStartE2EDuration="2.483775914s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:42:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:42:25.481494563 +0000 UTC m=+1127.687479959" watchObservedRunningTime="2026-09-08 18:42:25.483775914 +0000 UTC m=+1127.689761302" Sep 08 18:42:26.135221 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:26.135184 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:42:26.140211 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:26.140185 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:42:26.450497 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:26.450404 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:42:26.451585 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:26.451567 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:42:27.368106 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:27.368066 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:27.535149 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:27.535117 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:27.633785 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:27.633669 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:28.563116 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:28.563072 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:28.666075 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:28.666036 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:28.836062 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:28.835964 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:29.033485 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:29.033444 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:29.133188 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:29.133104 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:29.156161 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:29.156126 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6255f2-5b689d4b8b-j58cg"] Sep 08 18:42:29.161795 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:29.161767 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6255f2-5b689d4b8b-j58cg" Sep 08 18:42:29.164227 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:29.164201 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-6255f2-dockercfg-ckcmm\"" Sep 08 18:42:29.164227 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:29.164209 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-6255f2\"" Sep 08 18:42:29.168699 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:29.168663 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6255f2-5b689d4b8b-j58cg"] Sep 08 18:42:29.331425 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:29.331386 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72a1a9df-dd32-4a49-a5b5-72a4ce01b798-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-6255f2-5b689d4b8b-j58cg\" (UID: \"72a1a9df-dd32-4a49-a5b5-72a4ce01b798\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6255f2-5b689d4b8b-j58cg" Sep 08 18:42:29.331425 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:29.331428 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpdq6\" (UniqueName: \"kubernetes.io/projected/72a1a9df-dd32-4a49-a5b5-72a4ce01b798-kube-api-access-tpdq6\") pod \"payload-pre-processing-e2e-shared-b-w3-6255f2-5b689d4b8b-j58cg\" (UID: \"72a1a9df-dd32-4a49-a5b5-72a4ce01b798\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6255f2-5b689d4b8b-j58cg" Sep 08 18:42:29.432252 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:29.432162 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72a1a9df-dd32-4a49-a5b5-72a4ce01b798-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-6255f2-5b689d4b8b-j58cg\" (UID: \"72a1a9df-dd32-4a49-a5b5-72a4ce01b798\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6255f2-5b689d4b8b-j58cg" Sep 08 18:42:29.432252 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:29.432201 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpdq6\" (UniqueName: \"kubernetes.io/projected/72a1a9df-dd32-4a49-a5b5-72a4ce01b798-kube-api-access-tpdq6\") pod \"payload-pre-processing-e2e-shared-b-w3-6255f2-5b689d4b8b-j58cg\" (UID: \"72a1a9df-dd32-4a49-a5b5-72a4ce01b798\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6255f2-5b689d4b8b-j58cg" Sep 08 18:42:29.432863 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:29.432842 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72a1a9df-dd32-4a49-a5b5-72a4ce01b798-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-6255f2-5b689d4b8b-j58cg\" (UID: \"72a1a9df-dd32-4a49-a5b5-72a4ce01b798\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6255f2-5b689d4b8b-j58cg" Sep 08 18:42:29.440576 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:29.440555 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpdq6\" (UniqueName: \"kubernetes.io/projected/72a1a9df-dd32-4a49-a5b5-72a4ce01b798-kube-api-access-tpdq6\") pod \"payload-pre-processing-e2e-shared-b-w3-6255f2-5b689d4b8b-j58cg\" (UID: \"72a1a9df-dd32-4a49-a5b5-72a4ce01b798\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6255f2-5b689d4b8b-j58cg" Sep 08 18:42:29.474793 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:29.474764 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6255f2-5b689d4b8b-j58cg" Sep 08 18:42:29.745402 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:29.745371 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6255f2-5b689d4b8b-j58cg"] Sep 08 18:42:30.474224 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:30.474190 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6255f2-5b689d4b8b-j58cg" event={"ID":"72a1a9df-dd32-4a49-a5b5-72a4ce01b798","Type":"ContainerStarted","Data":"5737fe8def1fc2333a2cc65c112a0f7d51bf3ec31ef272fda43713c7a9b48f53"} Sep 08 18:42:30.474439 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:30.474248 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6255f2-5b689d4b8b-j58cg" event={"ID":"72a1a9df-dd32-4a49-a5b5-72a4ce01b798","Type":"ContainerStarted","Data":"41aefbf89cc741c4b3e495e72922f2aa19f892e2dd236f634832d6d59de36525"} Sep 08 18:42:30.474439 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:30.474359 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6255f2-5b689d4b8b-j58cg" Sep 08 18:42:30.492237 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:30.492185 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6255f2-5b689d4b8b-j58cg" podStartSLOduration=1.492170819 podStartE2EDuration="1.492170819s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:42:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:42:30.490737143 +0000 UTC m=+1132.696722539" watchObservedRunningTime="2026-09-08 18:42:30.492170819 +0000 UTC m=+1132.698156215" Sep 08 18:42:31.420198 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:31.420168 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b04d6-86d98c5cf8-jx6hr" Sep 08 18:42:31.420631 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:31.420354 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3b04d6-5f657ddf9f-6vk9h" Sep 08 18:42:33.433905 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:33.433869 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:42:36.918509 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:36.918476 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:39.722050 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:39.722013 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:41.483395 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:41.483359 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6255f2-5b689d4b8b-j58cg" Sep 08 18:42:42.287643 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:42.287612 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:42.491904 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:42.491861 2555 patch_prober.go:28] interesting pod/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 08 18:42:42.492299 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:42.491919 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" podUID="8ce8b8f9-b864-4072-bb13-92ad53c94917" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 08 18:42:42.619619 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:42.619529 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:42.918587 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:42.918493 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:43.133782 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:43.133710 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:43.561867 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:43.561834 2555 generic.go:356] "Generic (PLEG): container finished" podID="8ce8b8f9-b864-4072-bb13-92ad53c94917" containerID="56aeee8f913e923028600b7de31543839079e013ec480ddcabdd338d9cf23353" exitCode=0 Sep 08 18:42:43.562336 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:43.561891 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" event={"ID":"8ce8b8f9-b864-4072-bb13-92ad53c94917","Type":"ContainerDied","Data":"56aeee8f913e923028600b7de31543839079e013ec480ddcabdd338d9cf23353"} Sep 08 18:42:43.562402 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:43.562361 2555 scope.go:117] "RemoveContainer" containerID="56aeee8f913e923028600b7de31543839079e013ec480ddcabdd338d9cf23353" Sep 08 18:42:43.563427 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:43.563402 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 18:42:43.563524 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:43.563477 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 18:42:43.563588 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:43.563522 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 18:42:44.569580 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:44.569547 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" event={"ID":"8ce8b8f9-b864-4072-bb13-92ad53c94917","Type":"ContainerStarted","Data":"7c0ebfb8c60356bd0820461dee0e2e1dfc4b664b2ccfc9b78898615cd6a51efa"} Sep 08 18:42:45.134703 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:45.134658 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:42:45.134996 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:45.134965 2555 patch_prober.go:28] interesting pod/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Sep 08 18:42:45.135136 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:45.135015 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" podUID="8ce8b8f9-b864-4072-bb13-92ad53c94917" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Sep 08 18:42:45.559487 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:45.559443 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:45.649020 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:45.648985 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:46.135015 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:46.134983 2555 patch_prober.go:28] interesting pod/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Sep 08 18:42:46.135218 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:46.135040 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" podUID="8ce8b8f9-b864-4072-bb13-92ad53c94917" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Sep 08 18:42:46.761355 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:46.761325 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:46.862505 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:46.862470 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:47.134980 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:47.134947 2555 patch_prober.go:28] interesting pod/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Sep 08 18:42:47.135162 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:47.135006 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" podUID="8ce8b8f9-b864-4072-bb13-92ad53c94917" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Sep 08 18:42:47.295561 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:47.295511 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:47.407775 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:47.407250 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:47.420740 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:47.420713 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-951c3e-6657c964d6-l8bq9"] Sep 08 18:42:47.424403 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:47.424388 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-951c3e-6657c964d6-l8bq9" Sep 08 18:42:47.427451 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:47.427434 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-951c3e\"" Sep 08 18:42:47.432075 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:47.432052 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-951c3e-6869d8c4dc-tl7px"] Sep 08 18:42:47.435488 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:47.435468 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-951c3e-6869d8c4dc-tl7px" Sep 08 18:42:47.438279 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:47.438259 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-951c3e-dockercfg-tc8r6\"" Sep 08 18:42:47.439761 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:47.439739 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-951c3e-6657c964d6-l8bq9"] Sep 08 18:42:47.450805 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:47.450783 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-951c3e-6869d8c4dc-tl7px"] Sep 08 18:42:47.518750 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:47.518667 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ps96\" (UniqueName: \"kubernetes.io/projected/80c36c09-e533-446b-82fe-28d44a96e1d8-kube-api-access-8ps96\") pod \"payload-processing-e2e-shared-b-w2-951c3e-6869d8c4dc-tl7px\" (UID: \"80c36c09-e533-446b-82fe-28d44a96e1d8\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-951c3e-6869d8c4dc-tl7px" Sep 08 18:42:47.518943 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:47.518770 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/131d83ac-f065-488a-872b-c40c09cfccd5-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-951c3e-6657c964d6-l8bq9\" (UID: \"131d83ac-f065-488a-872b-c40c09cfccd5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-951c3e-6657c964d6-l8bq9" Sep 08 18:42:47.518943 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:47.518931 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80c36c09-e533-446b-82fe-28d44a96e1d8-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-951c3e-6869d8c4dc-tl7px\" (UID: \"80c36c09-e533-446b-82fe-28d44a96e1d8\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-951c3e-6869d8c4dc-tl7px" Sep 08 18:42:47.519078 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:47.519053 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjgdg\" (UniqueName: \"kubernetes.io/projected/131d83ac-f065-488a-872b-c40c09cfccd5-kube-api-access-xjgdg\") pod \"payload-pre-processing-e2e-shared-b-w2-951c3e-6657c964d6-l8bq9\" (UID: \"131d83ac-f065-488a-872b-c40c09cfccd5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-951c3e-6657c964d6-l8bq9" Sep 08 18:42:47.619959 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:47.619929 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/131d83ac-f065-488a-872b-c40c09cfccd5-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-951c3e-6657c964d6-l8bq9\" (UID: \"131d83ac-f065-488a-872b-c40c09cfccd5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-951c3e-6657c964d6-l8bq9" Sep 08 18:42:47.620141 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:47.620007 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80c36c09-e533-446b-82fe-28d44a96e1d8-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-951c3e-6869d8c4dc-tl7px\" (UID: \"80c36c09-e533-446b-82fe-28d44a96e1d8\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-951c3e-6869d8c4dc-tl7px" Sep 08 18:42:47.620141 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:47.620058 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjgdg\" (UniqueName: \"kubernetes.io/projected/131d83ac-f065-488a-872b-c40c09cfccd5-kube-api-access-xjgdg\") pod \"payload-pre-processing-e2e-shared-b-w2-951c3e-6657c964d6-l8bq9\" (UID: \"131d83ac-f065-488a-872b-c40c09cfccd5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-951c3e-6657c964d6-l8bq9" Sep 08 18:42:47.620141 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:47.620094 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ps96\" (UniqueName: \"kubernetes.io/projected/80c36c09-e533-446b-82fe-28d44a96e1d8-kube-api-access-8ps96\") pod \"payload-processing-e2e-shared-b-w2-951c3e-6869d8c4dc-tl7px\" (UID: \"80c36c09-e533-446b-82fe-28d44a96e1d8\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-951c3e-6869d8c4dc-tl7px" Sep 08 18:42:47.620624 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:47.620604 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/131d83ac-f065-488a-872b-c40c09cfccd5-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-951c3e-6657c964d6-l8bq9\" (UID: \"131d83ac-f065-488a-872b-c40c09cfccd5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-951c3e-6657c964d6-l8bq9" Sep 08 18:42:47.620805 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:47.620787 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80c36c09-e533-446b-82fe-28d44a96e1d8-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-951c3e-6869d8c4dc-tl7px\" (UID: \"80c36c09-e533-446b-82fe-28d44a96e1d8\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-951c3e-6869d8c4dc-tl7px" Sep 08 18:42:47.628563 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:47.628539 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ps96\" (UniqueName: \"kubernetes.io/projected/80c36c09-e533-446b-82fe-28d44a96e1d8-kube-api-access-8ps96\") pod \"payload-processing-e2e-shared-b-w2-951c3e-6869d8c4dc-tl7px\" (UID: \"80c36c09-e533-446b-82fe-28d44a96e1d8\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-951c3e-6869d8c4dc-tl7px" Sep 08 18:42:47.628672 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:47.628592 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjgdg\" (UniqueName: \"kubernetes.io/projected/131d83ac-f065-488a-872b-c40c09cfccd5-kube-api-access-xjgdg\") pod \"payload-pre-processing-e2e-shared-b-w2-951c3e-6657c964d6-l8bq9\" (UID: \"131d83ac-f065-488a-872b-c40c09cfccd5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-951c3e-6657c964d6-l8bq9" Sep 08 18:42:47.734849 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:47.734761 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-951c3e-6657c964d6-l8bq9" Sep 08 18:42:47.746087 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:47.746058 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-951c3e-6869d8c4dc-tl7px" Sep 08 18:42:48.033909 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:48.033883 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-951c3e-6657c964d6-l8bq9"] Sep 08 18:42:48.035046 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:42:48.035024 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod131d83ac_f065_488a_872b_c40c09cfccd5.slice/crio-b7c0695b56c0d9c60e73dead011af08b8640267728bafb5ae5f6c7515aab95b7 WatchSource:0}: Error finding container b7c0695b56c0d9c60e73dead011af08b8640267728bafb5ae5f6c7515aab95b7: Status 404 returned error can't find the container with id b7c0695b56c0d9c60e73dead011af08b8640267728bafb5ae5f6c7515aab95b7 Sep 08 18:42:48.061995 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:48.061974 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-951c3e-6869d8c4dc-tl7px"] Sep 08 18:42:48.073941 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:42:48.073915 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80c36c09_e533_446b_82fe_28d44a96e1d8.slice/crio-5a767960663f6d8a2149561b6c92a4ddf77b1b50894ef1a51a260b7b4e3a0c1e WatchSource:0}: Error finding container 5a767960663f6d8a2149561b6c92a4ddf77b1b50894ef1a51a260b7b4e3a0c1e: Status 404 returned error can't find the container with id 5a767960663f6d8a2149561b6c92a4ddf77b1b50894ef1a51a260b7b4e3a0c1e Sep 08 18:42:48.134807 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:48.134745 2555 patch_prober.go:28] interesting pod/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Sep 08 18:42:48.134939 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:48.134819 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" podUID="8ce8b8f9-b864-4072-bb13-92ad53c94917" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Sep 08 18:42:48.595776 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:48.595664 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-951c3e-6869d8c4dc-tl7px" event={"ID":"80c36c09-e533-446b-82fe-28d44a96e1d8","Type":"ContainerStarted","Data":"4ca09cf1ed95c220ac377c75d1187ca580f473ff042ca712e6d42ebfe3a11114"} Sep 08 18:42:48.595776 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:48.595743 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-951c3e-6869d8c4dc-tl7px" event={"ID":"80c36c09-e533-446b-82fe-28d44a96e1d8","Type":"ContainerStarted","Data":"5a767960663f6d8a2149561b6c92a4ddf77b1b50894ef1a51a260b7b4e3a0c1e"} Sep 08 18:42:48.596409 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:48.596380 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-951c3e-6869d8c4dc-tl7px" Sep 08 18:42:48.597779 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:48.597719 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-951c3e-6657c964d6-l8bq9" event={"ID":"131d83ac-f065-488a-872b-c40c09cfccd5","Type":"ContainerStarted","Data":"39c723be7c296a037f5e41bee8dc1e6b9e82532d0ff6f3bb562f379e2904fc5b"} Sep 08 18:42:48.597940 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:48.597904 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-951c3e-6657c964d6-l8bq9" event={"ID":"131d83ac-f065-488a-872b-c40c09cfccd5","Type":"ContainerStarted","Data":"b7c0695b56c0d9c60e73dead011af08b8640267728bafb5ae5f6c7515aab95b7"} Sep 08 18:42:48.617387 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:48.617325 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-951c3e-6869d8c4dc-tl7px" podStartSLOduration=1.617305266 podStartE2EDuration="1.617305266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:42:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:42:48.614001324 +0000 UTC m=+1150.819986720" watchObservedRunningTime="2026-09-08 18:42:48.617305266 +0000 UTC m=+1150.823290664" Sep 08 18:42:48.633115 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:48.633066 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-951c3e-6657c964d6-l8bq9" podStartSLOduration=1.633048493 podStartE2EDuration="1.633048493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:42:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:42:48.631548987 +0000 UTC m=+1150.837534391" watchObservedRunningTime="2026-09-08 18:42:48.633048493 +0000 UTC m=+1150.839033890" Sep 08 18:42:49.134293 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:49.134261 2555 patch_prober.go:28] interesting pod/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Sep 08 18:42:49.134672 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:49.134312 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" podUID="8ce8b8f9-b864-4072-bb13-92ad53c94917" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Sep 08 18:42:49.424365 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:49.424282 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:49.608143 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:49.608110 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-951c3e-6657c964d6-l8bq9" Sep 08 18:42:50.232664 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:50.232633 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:42:50.233086 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:50.233059 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:42:50.233878 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:50.233858 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:42:52.350435 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:52.350404 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:42:52.530953 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:52.530915 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:42:59.610529 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:42:59.610453 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-951c3e-6869d8c4dc-tl7px" Sep 08 18:43:00.615050 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:00.615021 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-951c3e-6657c964d6-l8bq9" Sep 08 18:43:02.871940 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:02.871904 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:43:08.730178 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:08.730143 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:43:10.532522 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:10.532494 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:43:11.661448 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:11.661414 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:43:15.334890 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:15.334848 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:43:15.565850 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:15.565811 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7575-02cafe"] Sep 08 18:43:15.569674 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:15.569650 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7575-02cafe" Sep 08 18:43:15.571882 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:15.571858 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-mhbkz\"" Sep 08 18:43:15.575313 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:15.575258 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7575-02cafe"] Sep 08 18:43:15.667807 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:15.667726 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgcb8\" (UniqueName: \"kubernetes.io/projected/f8e5c88d-895f-4c4b-afdc-c2ac688053af-kube-api-access-lgcb8\") pod \"test-curl-7575-02cafe\" (UID: \"f8e5c88d-895f-4c4b-afdc-c2ac688053af\") " pod="opendatahub/test-curl-7575-02cafe" Sep 08 18:43:15.769230 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:15.769197 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgcb8\" (UniqueName: \"kubernetes.io/projected/f8e5c88d-895f-4c4b-afdc-c2ac688053af-kube-api-access-lgcb8\") pod \"test-curl-7575-02cafe\" (UID: \"f8e5c88d-895f-4c4b-afdc-c2ac688053af\") " pod="opendatahub/test-curl-7575-02cafe" Sep 08 18:43:15.778129 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:15.778094 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgcb8\" (UniqueName: \"kubernetes.io/projected/f8e5c88d-895f-4c4b-afdc-c2ac688053af-kube-api-access-lgcb8\") pod \"test-curl-7575-02cafe\" (UID: \"f8e5c88d-895f-4c4b-afdc-c2ac688053af\") " pod="opendatahub/test-curl-7575-02cafe" Sep 08 18:43:15.881577 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:15.881537 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7575-02cafe" Sep 08 18:43:16.156653 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:16.156622 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7575-02cafe"] Sep 08 18:43:16.158075 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:43:16.158049 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf8e5c88d_895f_4c4b_afdc_c2ac688053af.slice/crio-b2c7ebbdd18a82fb73aa09db6a330ba880a959eab86eb81839ec6c853742eb28 WatchSource:0}: Error finding container b2c7ebbdd18a82fb73aa09db6a330ba880a959eab86eb81839ec6c853742eb28: Status 404 returned error can't find the container with id b2c7ebbdd18a82fb73aa09db6a330ba880a959eab86eb81839ec6c853742eb28 Sep 08 18:43:16.782628 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:16.782587 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7575-02cafe" event={"ID":"f8e5c88d-895f-4c4b-afdc-c2ac688053af","Type":"ContainerStarted","Data":"aba9b7a438bc360fb7a5df2b2ea6d9e09425d0029e8c3082ec6f2f5756cd917b"} Sep 08 18:43:16.783043 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:16.782641 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7575-02cafe" event={"ID":"f8e5c88d-895f-4c4b-afdc-c2ac688053af","Type":"ContainerStarted","Data":"b2c7ebbdd18a82fb73aa09db6a330ba880a959eab86eb81839ec6c853742eb28"} Sep 08 18:43:16.797703 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:16.797635 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7575-02cafe" podStartSLOduration=1.797619773 podStartE2EDuration="1.797619773s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:43:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:43:16.796007442 +0000 UTC m=+1179.001992838" watchObservedRunningTime="2026-09-08 18:43:16.797619773 +0000 UTC m=+1179.003605169" Sep 08 18:43:17.389799 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:17.389756 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7575-02cafe"] Sep 08 18:43:17.392381 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:17.392349 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7575-02cafe"] Sep 08 18:43:17.788007 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:43:17.787982 2555 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7575-02cafe" container="test-curl-7575-02cafe" Sep 08 18:43:18.793085 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:18.793044 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7575-02cafe" podUID="f8e5c88d-895f-4c4b-afdc-c2ac688053af" containerName="test-curl-7575-02cafe" containerID="cri-o://aba9b7a438bc360fb7a5df2b2ea6d9e09425d0029e8c3082ec6f2f5756cd917b" gracePeriod=2 Sep 08 18:43:20.062257 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:20.062227 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:43:20.379232 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:20.379153 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:43:20.576549 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:20.576512 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:43:20.971505 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:20.971474 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7575-02cafe" Sep 08 18:43:21.118134 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:21.118078 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lgcb8\" (UniqueName: \"kubernetes.io/projected/f8e5c88d-895f-4c4b-afdc-c2ac688053af-kube-api-access-lgcb8\") pod \"f8e5c88d-895f-4c4b-afdc-c2ac688053af\" (UID: \"f8e5c88d-895f-4c4b-afdc-c2ac688053af\") " Sep 08 18:43:21.120175 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:21.120145 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8e5c88d-895f-4c4b-afdc-c2ac688053af-kube-api-access-lgcb8" (OuterVolumeSpecName: "kube-api-access-lgcb8") pod "f8e5c88d-895f-4c4b-afdc-c2ac688053af" (UID: "f8e5c88d-895f-4c4b-afdc-c2ac688053af"). InnerVolumeSpecName "kube-api-access-lgcb8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:43:21.219136 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:21.219104 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lgcb8\" (UniqueName: \"kubernetes.io/projected/f8e5c88d-895f-4c4b-afdc-c2ac688053af-kube-api-access-lgcb8\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:43:21.812411 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:21.812380 2555 generic.go:356] "Generic (PLEG): container finished" podID="f8e5c88d-895f-4c4b-afdc-c2ac688053af" containerID="aba9b7a438bc360fb7a5df2b2ea6d9e09425d0029e8c3082ec6f2f5756cd917b" exitCode=137 Sep 08 18:43:21.812587 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:21.812431 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7575-02cafe" Sep 08 18:43:21.812587 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:21.812466 2555 scope.go:117] "RemoveContainer" containerID="aba9b7a438bc360fb7a5df2b2ea6d9e09425d0029e8c3082ec6f2f5756cd917b" Sep 08 18:43:21.821763 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:21.821739 2555 scope.go:117] "RemoveContainer" containerID="aba9b7a438bc360fb7a5df2b2ea6d9e09425d0029e8c3082ec6f2f5756cd917b" Sep 08 18:43:21.822059 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:43:21.822021 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aba9b7a438bc360fb7a5df2b2ea6d9e09425d0029e8c3082ec6f2f5756cd917b\": container with ID starting with aba9b7a438bc360fb7a5df2b2ea6d9e09425d0029e8c3082ec6f2f5756cd917b not found: ID does not exist" containerID="aba9b7a438bc360fb7a5df2b2ea6d9e09425d0029e8c3082ec6f2f5756cd917b" Sep 08 18:43:21.822059 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:21.822045 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aba9b7a438bc360fb7a5df2b2ea6d9e09425d0029e8c3082ec6f2f5756cd917b"} err="failed to get container status \"aba9b7a438bc360fb7a5df2b2ea6d9e09425d0029e8c3082ec6f2f5756cd917b\": rpc error: code = NotFound desc = could not find container \"aba9b7a438bc360fb7a5df2b2ea6d9e09425d0029e8c3082ec6f2f5756cd917b\": container with ID starting with aba9b7a438bc360fb7a5df2b2ea6d9e09425d0029e8c3082ec6f2f5756cd917b not found: ID does not exist" Sep 08 18:43:21.822192 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:43:21.822172 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7575-02cafe\" is forbidden: User \"system:node:ip-10-0-139-112.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-112.ec2.internal' and this object" podUID="f8e5c88d-895f-4c4b-afdc-c2ac688053af" pod="opendatahub/test-curl-7575-02cafe" Sep 08 18:43:22.324856 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:22.324826 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8e5c88d-895f-4c4b-afdc-c2ac688053af" path="/var/lib/kubelet/pods/f8e5c88d-895f-4c4b-afdc-c2ac688053af/volumes" Sep 08 18:43:23.840418 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:23.840378 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:43:23.942667 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:23.942632 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:43:25.007591 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:25.007560 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:43:25.107433 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:25.107399 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:43:25.207813 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:25.207783 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:43:25.603492 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:25.603442 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:43:25.698739 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:25.698708 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:43:25.706941 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:25.706916 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-783195cb-574777cd78-f25lk"] Sep 08 18:43:25.707362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:25.707350 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8e5c88d-895f-4c4b-afdc-c2ac688053af" containerName="test-curl-7575-02cafe" Sep 08 18:43:25.707408 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:25.707364 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8e5c88d-895f-4c4b-afdc-c2ac688053af" containerName="test-curl-7575-02cafe" Sep 08 18:43:25.707461 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:25.707451 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8e5c88d-895f-4c4b-afdc-c2ac688053af" containerName="test-curl-7575-02cafe" Sep 08 18:43:25.712028 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:25.712012 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-783195cb-574777cd78-f25lk" Sep 08 18:43:25.714301 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:25.714280 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-783195cb\"" Sep 08 18:43:25.722114 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:25.722090 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-783195cb-574777cd78-f25lk"] Sep 08 18:43:25.735205 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:25.735179 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-783195cb-7c4d68689-9v79z"] Sep 08 18:43:25.738991 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:25.738974 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-783195cb-7c4d68689-9v79z" Sep 08 18:43:25.741491 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:25.741471 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-783195cb-dockercfg-gkgsz\"" Sep 08 18:43:25.750107 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:25.750088 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-783195cb-7c4d68689-9v79z"] Sep 08 18:43:25.862035 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:25.861946 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac9c5225-28db-4893-8806-22272084fdb4-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-783195cb-574777cd78-f25lk\" (UID: \"ac9c5225-28db-4893-8806-22272084fdb4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-783195cb-574777cd78-f25lk" Sep 08 18:43:25.862035 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:25.861984 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q55xq\" (UniqueName: \"kubernetes.io/projected/62ba8b2d-323c-4f88-8fc4-d9d0e954b847-kube-api-access-q55xq\") pod \"payload-processing-e2e-mt-783195cb-7c4d68689-9v79z\" (UID: \"62ba8b2d-323c-4f88-8fc4-d9d0e954b847\") " pod="openshift-ingress/payload-processing-e2e-mt-783195cb-7c4d68689-9v79z" Sep 08 18:43:25.862221 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:25.862064 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62ba8b2d-323c-4f88-8fc4-d9d0e954b847-plugins-config-volume\") pod \"payload-processing-e2e-mt-783195cb-7c4d68689-9v79z\" (UID: \"62ba8b2d-323c-4f88-8fc4-d9d0e954b847\") " pod="openshift-ingress/payload-processing-e2e-mt-783195cb-7c4d68689-9v79z" Sep 08 18:43:25.862221 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:25.862117 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbmp9\" (UniqueName: \"kubernetes.io/projected/ac9c5225-28db-4893-8806-22272084fdb4-kube-api-access-cbmp9\") pod \"payload-pre-processing-e2e-mt-783195cb-574777cd78-f25lk\" (UID: \"ac9c5225-28db-4893-8806-22272084fdb4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-783195cb-574777cd78-f25lk" Sep 08 18:43:25.962920 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:25.962870 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbmp9\" (UniqueName: \"kubernetes.io/projected/ac9c5225-28db-4893-8806-22272084fdb4-kube-api-access-cbmp9\") pod \"payload-pre-processing-e2e-mt-783195cb-574777cd78-f25lk\" (UID: \"ac9c5225-28db-4893-8806-22272084fdb4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-783195cb-574777cd78-f25lk" Sep 08 18:43:25.963126 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:25.962955 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac9c5225-28db-4893-8806-22272084fdb4-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-783195cb-574777cd78-f25lk\" (UID: \"ac9c5225-28db-4893-8806-22272084fdb4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-783195cb-574777cd78-f25lk" Sep 08 18:43:25.963126 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:25.962975 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q55xq\" (UniqueName: \"kubernetes.io/projected/62ba8b2d-323c-4f88-8fc4-d9d0e954b847-kube-api-access-q55xq\") pod \"payload-processing-e2e-mt-783195cb-7c4d68689-9v79z\" (UID: \"62ba8b2d-323c-4f88-8fc4-d9d0e954b847\") " pod="openshift-ingress/payload-processing-e2e-mt-783195cb-7c4d68689-9v79z" Sep 08 18:43:25.963126 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:25.963024 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62ba8b2d-323c-4f88-8fc4-d9d0e954b847-plugins-config-volume\") pod \"payload-processing-e2e-mt-783195cb-7c4d68689-9v79z\" (UID: \"62ba8b2d-323c-4f88-8fc4-d9d0e954b847\") " pod="openshift-ingress/payload-processing-e2e-mt-783195cb-7c4d68689-9v79z" Sep 08 18:43:25.963679 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:25.963659 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62ba8b2d-323c-4f88-8fc4-d9d0e954b847-plugins-config-volume\") pod \"payload-processing-e2e-mt-783195cb-7c4d68689-9v79z\" (UID: \"62ba8b2d-323c-4f88-8fc4-d9d0e954b847\") " pod="openshift-ingress/payload-processing-e2e-mt-783195cb-7c4d68689-9v79z" Sep 08 18:43:25.963784 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:25.963674 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac9c5225-28db-4893-8806-22272084fdb4-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-783195cb-574777cd78-f25lk\" (UID: \"ac9c5225-28db-4893-8806-22272084fdb4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-783195cb-574777cd78-f25lk" Sep 08 18:43:25.972909 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:25.972884 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q55xq\" (UniqueName: \"kubernetes.io/projected/62ba8b2d-323c-4f88-8fc4-d9d0e954b847-kube-api-access-q55xq\") pod \"payload-processing-e2e-mt-783195cb-7c4d68689-9v79z\" (UID: \"62ba8b2d-323c-4f88-8fc4-d9d0e954b847\") " pod="openshift-ingress/payload-processing-e2e-mt-783195cb-7c4d68689-9v79z" Sep 08 18:43:25.973007 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:25.972993 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbmp9\" (UniqueName: \"kubernetes.io/projected/ac9c5225-28db-4893-8806-22272084fdb4-kube-api-access-cbmp9\") pod \"payload-pre-processing-e2e-mt-783195cb-574777cd78-f25lk\" (UID: \"ac9c5225-28db-4893-8806-22272084fdb4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-783195cb-574777cd78-f25lk" Sep 08 18:43:26.023972 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:26.023938 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-783195cb-574777cd78-f25lk" Sep 08 18:43:26.049130 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:26.049100 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-783195cb-7c4d68689-9v79z" Sep 08 18:43:26.302330 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:26.302307 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-783195cb-574777cd78-f25lk"] Sep 08 18:43:26.303808 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:43:26.303782 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac9c5225_28db_4893_8806_22272084fdb4.slice/crio-090d24a7583c3b02eb2abb5bf19c4ed4beca4ff50682e8f8a1dc6389ac3f5125 WatchSource:0}: Error finding container 090d24a7583c3b02eb2abb5bf19c4ed4beca4ff50682e8f8a1dc6389ac3f5125: Status 404 returned error can't find the container with id 090d24a7583c3b02eb2abb5bf19c4ed4beca4ff50682e8f8a1dc6389ac3f5125 Sep 08 18:43:26.334218 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:26.334191 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-783195cb-7c4d68689-9v79z"] Sep 08 18:43:26.339851 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:43:26.339824 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62ba8b2d_323c_4f88_8fc4_d9d0e954b847.slice/crio-d63177c6a60a2bf49a66c57d40341767e595e5e182ca7b7be96330f03c656d3d WatchSource:0}: Error finding container d63177c6a60a2bf49a66c57d40341767e595e5e182ca7b7be96330f03c656d3d: Status 404 returned error can't find the container with id d63177c6a60a2bf49a66c57d40341767e595e5e182ca7b7be96330f03c656d3d Sep 08 18:43:26.844060 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:26.844026 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-783195cb-574777cd78-f25lk" event={"ID":"ac9c5225-28db-4893-8806-22272084fdb4","Type":"ContainerStarted","Data":"4b6c4679455331ecdf3c2f920fdc66a064a166d781ea3ebed4e886be3236b277"} Sep 08 18:43:26.844235 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:26.844092 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-783195cb-574777cd78-f25lk" Sep 08 18:43:26.844235 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:26.844114 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-783195cb-574777cd78-f25lk" event={"ID":"ac9c5225-28db-4893-8806-22272084fdb4","Type":"ContainerStarted","Data":"090d24a7583c3b02eb2abb5bf19c4ed4beca4ff50682e8f8a1dc6389ac3f5125"} Sep 08 18:43:26.845576 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:26.845548 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-783195cb-7c4d68689-9v79z" event={"ID":"62ba8b2d-323c-4f88-8fc4-d9d0e954b847","Type":"ContainerStarted","Data":"a4223c9f22e1f9ba4fa5686b5073d7c978d8475296959c0926b7322e93f8bc35"} Sep 08 18:43:26.845711 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:26.845594 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-783195cb-7c4d68689-9v79z" event={"ID":"62ba8b2d-323c-4f88-8fc4-d9d0e954b847","Type":"ContainerStarted","Data":"d63177c6a60a2bf49a66c57d40341767e595e5e182ca7b7be96330f03c656d3d"} Sep 08 18:43:26.845711 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:26.845622 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-783195cb-7c4d68689-9v79z" Sep 08 18:43:26.863365 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:26.863313 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-783195cb-574777cd78-f25lk" podStartSLOduration=1.8632989549999999 podStartE2EDuration="1.863298955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:43:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:43:26.860704968 +0000 UTC m=+1189.066690358" watchObservedRunningTime="2026-09-08 18:43:26.863298955 +0000 UTC m=+1189.069284350" Sep 08 18:43:26.880402 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:26.880357 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-783195cb-7c4d68689-9v79z" podStartSLOduration=1.8803420549999998 podStartE2EDuration="1.880342055s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:43:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:43:26.877519098 +0000 UTC m=+1189.083504488" watchObservedRunningTime="2026-09-08 18:43:26.880342055 +0000 UTC m=+1189.086327452" Sep 08 18:43:32.640041 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:32.639999 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:43:37.456470 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:37.456425 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:43:37.560991 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:37.560962 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:43:37.853529 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:37.853503 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-783195cb-574777cd78-f25lk" Sep 08 18:43:37.853716 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:37.853660 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-783195cb-7c4d68689-9v79z" Sep 08 18:43:38.378223 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:38.378197 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9n28f_911c61b8-766b-4ae8-b046-784e22e85889/console-operator/1.log" Sep 08 18:43:38.381665 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:38.381639 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9n28f_911c61b8-766b-4ae8-b046-784e22e85889/console-operator/1.log" Sep 08 18:43:38.652964 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:38.652888 2555 scope.go:117] "RemoveContainer" containerID="6e1005d8628e72ff4a1566be45b23c3bfd17a702009f6a2b97b7911a2d013d1a" Sep 08 18:43:40.150707 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:40.150653 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:43:44.329226 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:44.329200 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:43:48.761188 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:48.761151 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:43:51.250250 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:43:51.250220 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:44:06.131813 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:06.131784 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:44:17.795040 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:17.795006 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-783195cb-wwlm9"] Sep 08 18:44:17.798624 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:17.798609 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-783195cb-wwlm9" Sep 08 18:44:17.800743 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:17.800721 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 08 18:44:17.800852 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:17.800764 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g6q8h\"" Sep 08 18:44:17.806659 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:17.806636 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-783195cb-wwlm9"] Sep 08 18:44:17.834451 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:17.834427 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:44:17.915657 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:17.915633 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28234074-eca4-4021-9236-61fc3f97fcbe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-783195cb-wwlm9\" (UID: \"28234074-eca4-4021-9236-61fc3f97fcbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-783195cb-wwlm9" Sep 08 18:44:18.016332 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:18.016309 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28234074-eca4-4021-9236-61fc3f97fcbe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-783195cb-wwlm9\" (UID: \"28234074-eca4-4021-9236-61fc3f97fcbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-783195cb-wwlm9" Sep 08 18:44:18.016858 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:18.016839 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28234074-eca4-4021-9236-61fc3f97fcbe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-783195cb-wwlm9\" (UID: \"28234074-eca4-4021-9236-61fc3f97fcbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-783195cb-wwlm9" Sep 08 18:44:18.109972 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:18.109925 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-783195cb-wwlm9" Sep 08 18:44:18.381870 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:18.381843 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-783195cb-wwlm9"] Sep 08 18:44:18.382875 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:44:18.382851 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28234074_eca4_4021_9236_61fc3f97fcbe.slice/crio-e9a0c9cecd56ab4039c182506d7c998e3db269ac689d01e427a41ffbc08b4bd2 WatchSource:0}: Error finding container e9a0c9cecd56ab4039c182506d7c998e3db269ac689d01e427a41ffbc08b4bd2: Status 404 returned error can't find the container with id e9a0c9cecd56ab4039c182506d7c998e3db269ac689d01e427a41ffbc08b4bd2 Sep 08 18:44:19.151729 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:19.151701 2555 generic.go:356] "Generic (PLEG): container finished" podID="28234074-eca4-4021-9236-61fc3f97fcbe" containerID="8419462dee219579546ee58bc7e3b04daf483e225b6e0a8b2c6cd5a8c7085da3" exitCode=0 Sep 08 18:44:19.152041 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:19.151782 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-783195cb-wwlm9" event={"ID":"28234074-eca4-4021-9236-61fc3f97fcbe","Type":"ContainerDied","Data":"8419462dee219579546ee58bc7e3b04daf483e225b6e0a8b2c6cd5a8c7085da3"} Sep 08 18:44:19.152041 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:19.151834 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-783195cb-wwlm9" event={"ID":"28234074-eca4-4021-9236-61fc3f97fcbe","Type":"ContainerStarted","Data":"e9a0c9cecd56ab4039c182506d7c998e3db269ac689d01e427a41ffbc08b4bd2"} Sep 08 18:44:20.288329 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:20.288307 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-783195cb-wwlm9" Sep 08 18:44:20.435812 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:20.435747 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28234074-eca4-4021-9236-61fc3f97fcbe-maas-api-service-ca\") pod \"28234074-eca4-4021-9236-61fc3f97fcbe\" (UID: \"28234074-eca4-4021-9236-61fc3f97fcbe\") " Sep 08 18:44:20.436042 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:20.436011 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28234074-eca4-4021-9236-61fc3f97fcbe-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "28234074-eca4-4021-9236-61fc3f97fcbe" (UID: "28234074-eca4-4021-9236-61fc3f97fcbe"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:44:20.536724 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:20.536702 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28234074-eca4-4021-9236-61fc3f97fcbe-maas-api-service-ca\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:44:21.164467 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:21.164435 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-783195cb-wwlm9" Sep 08 18:44:21.164623 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:21.164486 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-783195cb-wwlm9" event={"ID":"28234074-eca4-4021-9236-61fc3f97fcbe","Type":"ContainerDied","Data":"e9a0c9cecd56ab4039c182506d7c998e3db269ac689d01e427a41ffbc08b4bd2"} Sep 08 18:44:21.164623 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:21.164553 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e9a0c9cecd56ab4039c182506d7c998e3db269ac689d01e427a41ffbc08b4bd2" Sep 08 18:44:30.259270 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:30.259188 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:44:33.506937 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:33.506899 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:44:33.535451 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:33.535418 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-783195cb-7c4d68689-9v79z"] Sep 08 18:44:33.535731 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:33.535662 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-783195cb-7c4d68689-9v79z" podUID="62ba8b2d-323c-4f88-8fc4-d9d0e954b847" containerName="payload-processing" containerID="cri-o://a4223c9f22e1f9ba4fa5686b5073d7c978d8475296959c0926b7322e93f8bc35" gracePeriod=30 Sep 08 18:44:33.544563 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:33.544535 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-783195cb-574777cd78-f25lk"] Sep 08 18:44:33.544963 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:33.544933 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-783195cb-574777cd78-f25lk" podUID="ac9c5225-28db-4893-8806-22272084fdb4" containerName="payload-pre-processing" containerID="cri-o://4b6c4679455331ecdf3c2f920fdc66a064a166d781ea3ebed4e886be3236b277" gracePeriod=30 Sep 08 18:44:33.607025 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:33.606995 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:44:33.707919 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:33.707475 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:44:33.717101 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:33.717077 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-783195cb-wwlm9"] Sep 08 18:44:33.721286 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:33.721265 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-783195cb-wwlm9"] Sep 08 18:44:33.777126 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:33.777038 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:44:33.818289 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:33.818239 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-783195cb-7c4d68689-9v79z" Sep 08 18:44:33.821045 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:33.821028 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-783195cb-574777cd78-f25lk" Sep 08 18:44:33.942968 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:33.942934 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbmp9\" (UniqueName: \"kubernetes.io/projected/ac9c5225-28db-4893-8806-22272084fdb4-kube-api-access-cbmp9\") pod \"ac9c5225-28db-4893-8806-22272084fdb4\" (UID: \"ac9c5225-28db-4893-8806-22272084fdb4\") " Sep 08 18:44:33.943146 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:33.943023 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q55xq\" (UniqueName: \"kubernetes.io/projected/62ba8b2d-323c-4f88-8fc4-d9d0e954b847-kube-api-access-q55xq\") pod \"62ba8b2d-323c-4f88-8fc4-d9d0e954b847\" (UID: \"62ba8b2d-323c-4f88-8fc4-d9d0e954b847\") " Sep 08 18:44:33.943146 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:33.943110 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac9c5225-28db-4893-8806-22272084fdb4-plugins-config-volume\") pod \"ac9c5225-28db-4893-8806-22272084fdb4\" (UID: \"ac9c5225-28db-4893-8806-22272084fdb4\") " Sep 08 18:44:33.943146 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:33.943137 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62ba8b2d-323c-4f88-8fc4-d9d0e954b847-plugins-config-volume\") pod \"62ba8b2d-323c-4f88-8fc4-d9d0e954b847\" (UID: \"62ba8b2d-323c-4f88-8fc4-d9d0e954b847\") " Sep 08 18:44:33.943539 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:33.943506 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac9c5225-28db-4893-8806-22272084fdb4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ac9c5225-28db-4893-8806-22272084fdb4" (UID: "ac9c5225-28db-4893-8806-22272084fdb4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:44:33.943539 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:33.943508 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62ba8b2d-323c-4f88-8fc4-d9d0e954b847-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "62ba8b2d-323c-4f88-8fc4-d9d0e954b847" (UID: "62ba8b2d-323c-4f88-8fc4-d9d0e954b847"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:44:33.945310 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:33.945288 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac9c5225-28db-4893-8806-22272084fdb4-kube-api-access-cbmp9" (OuterVolumeSpecName: "kube-api-access-cbmp9") pod "ac9c5225-28db-4893-8806-22272084fdb4" (UID: "ac9c5225-28db-4893-8806-22272084fdb4"). InnerVolumeSpecName "kube-api-access-cbmp9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:44:33.945403 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:33.945344 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62ba8b2d-323c-4f88-8fc4-d9d0e954b847-kube-api-access-q55xq" (OuterVolumeSpecName: "kube-api-access-q55xq") pod "62ba8b2d-323c-4f88-8fc4-d9d0e954b847" (UID: "62ba8b2d-323c-4f88-8fc4-d9d0e954b847"). InnerVolumeSpecName "kube-api-access-q55xq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:44:34.044324 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:34.044293 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q55xq\" (UniqueName: \"kubernetes.io/projected/62ba8b2d-323c-4f88-8fc4-d9d0e954b847-kube-api-access-q55xq\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:44:34.044324 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:34.044324 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac9c5225-28db-4893-8806-22272084fdb4-plugins-config-volume\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:44:34.044507 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:34.044333 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62ba8b2d-323c-4f88-8fc4-d9d0e954b847-plugins-config-volume\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:44:34.044507 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:34.044342 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbmp9\" (UniqueName: \"kubernetes.io/projected/ac9c5225-28db-4893-8806-22272084fdb4-kube-api-access-cbmp9\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:44:34.242866 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:34.242827 2555 generic.go:356] "Generic (PLEG): container finished" podID="ac9c5225-28db-4893-8806-22272084fdb4" containerID="4b6c4679455331ecdf3c2f920fdc66a064a166d781ea3ebed4e886be3236b277" exitCode=0 Sep 08 18:44:34.242866 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:34.242871 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-783195cb-574777cd78-f25lk" Sep 08 18:44:34.243134 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:34.242917 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-783195cb-574777cd78-f25lk" event={"ID":"ac9c5225-28db-4893-8806-22272084fdb4","Type":"ContainerDied","Data":"4b6c4679455331ecdf3c2f920fdc66a064a166d781ea3ebed4e886be3236b277"} Sep 08 18:44:34.243134 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:34.242975 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-783195cb-574777cd78-f25lk" event={"ID":"ac9c5225-28db-4893-8806-22272084fdb4","Type":"ContainerDied","Data":"090d24a7583c3b02eb2abb5bf19c4ed4beca4ff50682e8f8a1dc6389ac3f5125"} Sep 08 18:44:34.243134 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:34.243005 2555 scope.go:117] "RemoveContainer" containerID="4b6c4679455331ecdf3c2f920fdc66a064a166d781ea3ebed4e886be3236b277" Sep 08 18:44:34.244440 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:34.244415 2555 generic.go:356] "Generic (PLEG): container finished" podID="62ba8b2d-323c-4f88-8fc4-d9d0e954b847" containerID="a4223c9f22e1f9ba4fa5686b5073d7c978d8475296959c0926b7322e93f8bc35" exitCode=0 Sep 08 18:44:34.244571 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:34.244460 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-783195cb-7c4d68689-9v79z" Sep 08 18:44:34.244571 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:34.244477 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-783195cb-7c4d68689-9v79z" event={"ID":"62ba8b2d-323c-4f88-8fc4-d9d0e954b847","Type":"ContainerDied","Data":"a4223c9f22e1f9ba4fa5686b5073d7c978d8475296959c0926b7322e93f8bc35"} Sep 08 18:44:34.244705 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:34.244671 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-783195cb-7c4d68689-9v79z" event={"ID":"62ba8b2d-323c-4f88-8fc4-d9d0e954b847","Type":"ContainerDied","Data":"d63177c6a60a2bf49a66c57d40341767e595e5e182ca7b7be96330f03c656d3d"} Sep 08 18:44:34.254124 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:34.254036 2555 scope.go:117] "RemoveContainer" containerID="4b6c4679455331ecdf3c2f920fdc66a064a166d781ea3ebed4e886be3236b277" Sep 08 18:44:34.254423 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:44:34.254400 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b6c4679455331ecdf3c2f920fdc66a064a166d781ea3ebed4e886be3236b277\": container with ID starting with 4b6c4679455331ecdf3c2f920fdc66a064a166d781ea3ebed4e886be3236b277 not found: ID does not exist" containerID="4b6c4679455331ecdf3c2f920fdc66a064a166d781ea3ebed4e886be3236b277" Sep 08 18:44:34.254534 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:34.254429 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b6c4679455331ecdf3c2f920fdc66a064a166d781ea3ebed4e886be3236b277"} err="failed to get container status \"4b6c4679455331ecdf3c2f920fdc66a064a166d781ea3ebed4e886be3236b277\": rpc error: code = NotFound desc = could not find container \"4b6c4679455331ecdf3c2f920fdc66a064a166d781ea3ebed4e886be3236b277\": container with ID starting with 4b6c4679455331ecdf3c2f920fdc66a064a166d781ea3ebed4e886be3236b277 not found: ID does not exist" Sep 08 18:44:34.254534 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:34.254446 2555 scope.go:117] "RemoveContainer" containerID="a4223c9f22e1f9ba4fa5686b5073d7c978d8475296959c0926b7322e93f8bc35" Sep 08 18:44:34.265702 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:34.265665 2555 scope.go:117] "RemoveContainer" containerID="a4223c9f22e1f9ba4fa5686b5073d7c978d8475296959c0926b7322e93f8bc35" Sep 08 18:44:34.266382 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:44:34.266360 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4223c9f22e1f9ba4fa5686b5073d7c978d8475296959c0926b7322e93f8bc35\": container with ID starting with a4223c9f22e1f9ba4fa5686b5073d7c978d8475296959c0926b7322e93f8bc35 not found: ID does not exist" containerID="a4223c9f22e1f9ba4fa5686b5073d7c978d8475296959c0926b7322e93f8bc35" Sep 08 18:44:34.266462 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:34.266390 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4223c9f22e1f9ba4fa5686b5073d7c978d8475296959c0926b7322e93f8bc35"} err="failed to get container status \"a4223c9f22e1f9ba4fa5686b5073d7c978d8475296959c0926b7322e93f8bc35\": rpc error: code = NotFound desc = could not find container \"a4223c9f22e1f9ba4fa5686b5073d7c978d8475296959c0926b7322e93f8bc35\": container with ID starting with a4223c9f22e1f9ba4fa5686b5073d7c978d8475296959c0926b7322e93f8bc35 not found: ID does not exist" Sep 08 18:44:34.268017 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:34.267995 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-783195cb-574777cd78-f25lk"] Sep 08 18:44:34.270898 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:34.270878 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-783195cb-574777cd78-f25lk"] Sep 08 18:44:34.281213 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:34.281192 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-783195cb-7c4d68689-9v79z"] Sep 08 18:44:34.285809 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:34.285792 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-783195cb-7c4d68689-9v79z"] Sep 08 18:44:34.324833 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:34.324762 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28234074-eca4-4021-9236-61fc3f97fcbe" path="/var/lib/kubelet/pods/28234074-eca4-4021-9236-61fc3f97fcbe/volumes" Sep 08 18:44:34.325190 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:34.325176 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62ba8b2d-323c-4f88-8fc4-d9d0e954b847" path="/var/lib/kubelet/pods/62ba8b2d-323c-4f88-8fc4-d9d0e954b847/volumes" Sep 08 18:44:34.325484 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:34.325472 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac9c5225-28db-4893-8806-22272084fdb4" path="/var/lib/kubelet/pods/ac9c5225-28db-4893-8806-22272084fdb4/volumes" Sep 08 18:44:36.771053 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:36.771023 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:44:37.534506 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:37.534473 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:44:41.850483 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:41.850445 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:44:42.257243 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:42.257163 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:44:55.330484 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:44:55.330449 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:45:00.148013 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:00.147973 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6255f2-29814885-7g47b"] Sep 08 18:45:00.148454 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:00.148395 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62ba8b2d-323c-4f88-8fc4-d9d0e954b847" containerName="payload-processing" Sep 08 18:45:00.148454 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:00.148408 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="62ba8b2d-323c-4f88-8fc4-d9d0e954b847" containerName="payload-processing" Sep 08 18:45:00.148454 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:00.148422 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28234074-eca4-4021-9236-61fc3f97fcbe" containerName="revoke-keys" Sep 08 18:45:00.148454 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:00.148431 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="28234074-eca4-4021-9236-61fc3f97fcbe" containerName="revoke-keys" Sep 08 18:45:00.148454 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:00.148442 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac9c5225-28db-4893-8806-22272084fdb4" containerName="payload-pre-processing" Sep 08 18:45:00.148454 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:00.148448 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac9c5225-28db-4893-8806-22272084fdb4" containerName="payload-pre-processing" Sep 08 18:45:00.148791 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:00.148527 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac9c5225-28db-4893-8806-22272084fdb4" containerName="payload-pre-processing" Sep 08 18:45:00.148791 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:00.148535 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="62ba8b2d-323c-4f88-8fc4-d9d0e954b847" containerName="payload-processing" Sep 08 18:45:00.148791 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:00.148543 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="28234074-eca4-4021-9236-61fc3f97fcbe" containerName="revoke-keys" Sep 08 18:45:00.152949 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:00.152930 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-3b04d6-29814885-vqkhv"] Sep 08 18:45:00.153082 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:00.153050 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6255f2-29814885-7g47b" Sep 08 18:45:00.155459 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:00.155438 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g6q8h\"" Sep 08 18:45:00.156290 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:00.156272 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-792c35-29814885-ttblp"] Sep 08 18:45:00.156475 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:00.156452 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-3b04d6-29814885-vqkhv" Sep 08 18:45:00.162209 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:00.161544 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6255f2-29814885-7g47b"] Sep 08 18:45:00.162209 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:00.161630 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-792c35-29814885-ttblp" Sep 08 18:45:00.164177 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:00.164156 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-792c35-29814885-ttblp"] Sep 08 18:45:00.167129 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:00.167107 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6255f2-29814885-7g47b" Sep 08 18:45:00.176105 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:00.176087 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-3b04d6-29814885-vqkhv" Sep 08 18:45:00.186384 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:00.186317 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-3b04d6-29814885-vqkhv"] Sep 08 18:45:00.218546 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:00.218522 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-792c35-29814885-ttblp" Sep 08 18:45:00.461622 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:00.461599 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6255f2-29814885-7g47b"] Sep 08 18:45:00.463082 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:45:00.463059 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33a34f27_be15_4794_9d9d_1250d3df9570.slice/crio-420a3dd164f3f8e55391263ead324e7b047078b12b1ff6154cac04e8d3beddba WatchSource:0}: Error finding container 420a3dd164f3f8e55391263ead324e7b047078b12b1ff6154cac04e8d3beddba: Status 404 returned error can't find the container with id 420a3dd164f3f8e55391263ead324e7b047078b12b1ff6154cac04e8d3beddba Sep 08 18:45:00.472374 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:00.472347 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-3b04d6-29814885-vqkhv"] Sep 08 18:45:00.472458 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:45:00.472355 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda3000a7_1ea1_4009_97e9_62483f7d70db.slice/crio-a9b3e50223aedaa5d6b238ef5e28381be77014757045ca97ead4046495e443ed WatchSource:0}: Error finding container a9b3e50223aedaa5d6b238ef5e28381be77014757045ca97ead4046495e443ed: Status 404 returned error can't find the container with id a9b3e50223aedaa5d6b238ef5e28381be77014757045ca97ead4046495e443ed Sep 08 18:45:00.513544 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:00.513513 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-792c35-29814885-ttblp"] Sep 08 18:45:00.514521 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:45:00.514500 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4eff982_06b0_410a_887d_b4d4cb85336e.slice/crio-90b1390e18d796eccaa878b8266d02676f0c0a50be3efce746c9db506c9792ba WatchSource:0}: Error finding container 90b1390e18d796eccaa878b8266d02676f0c0a50be3efce746c9db506c9792ba: Status 404 returned error can't find the container with id 90b1390e18d796eccaa878b8266d02676f0c0a50be3efce746c9db506c9792ba Sep 08 18:45:01.399410 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:01.399378 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-3b04d6-29814885-vqkhv" event={"ID":"da3000a7-1ea1-4009-97e9-62483f7d70db","Type":"ContainerStarted","Data":"a9a5238a98d6168e5b3c4999538e60e9446450d22b61f6ded9ec9e4cc66a23d6"} Sep 08 18:45:01.399768 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:01.399437 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-3b04d6-29814885-vqkhv" event={"ID":"da3000a7-1ea1-4009-97e9-62483f7d70db","Type":"ContainerStarted","Data":"a9b3e50223aedaa5d6b238ef5e28381be77014757045ca97ead4046495e443ed"} Sep 08 18:45:01.400986 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:01.400966 2555 generic.go:356] "Generic (PLEG): container finished" podID="f4eff982-06b0-410a-887d-b4d4cb85336e" containerID="c865b21c6bc7f255ab17f662a28fb7829cf1e29aeb04ebe9444de8394d444564" exitCode=0 Sep 08 18:45:01.401074 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:01.401034 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-792c35-29814885-ttblp" event={"ID":"f4eff982-06b0-410a-887d-b4d4cb85336e","Type":"ContainerDied","Data":"c865b21c6bc7f255ab17f662a28fb7829cf1e29aeb04ebe9444de8394d444564"} Sep 08 18:45:01.401074 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:01.401072 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-792c35-29814885-ttblp" event={"ID":"f4eff982-06b0-410a-887d-b4d4cb85336e","Type":"ContainerStarted","Data":"90b1390e18d796eccaa878b8266d02676f0c0a50be3efce746c9db506c9792ba"} Sep 08 18:45:01.402570 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:01.402550 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6255f2-29814885-7g47b" event={"ID":"33a34f27-be15-4794-9d9d-1250d3df9570","Type":"ContainerStarted","Data":"03737ea2c427e69c29f1aeb1e1f53c2ce7000cb7b77ff59b21ca19711b05c51d"} Sep 08 18:45:01.402665 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:01.402592 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6255f2-29814885-7g47b" event={"ID":"33a34f27-be15-4794-9d9d-1250d3df9570","Type":"ContainerStarted","Data":"420a3dd164f3f8e55391263ead324e7b047078b12b1ff6154cac04e8d3beddba"} Sep 08 18:45:01.416450 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:01.416412 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-3b04d6-29814885-vqkhv" podStartSLOduration=1.4164000030000001 podStartE2EDuration="1.416400003s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:45:01.414814624 +0000 UTC m=+1283.620800021" watchObservedRunningTime="2026-09-08 18:45:01.416400003 +0000 UTC m=+1283.622385533" Sep 08 18:45:01.434952 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:01.434907 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6255f2-29814885-7g47b" podStartSLOduration=1.434895493 podStartE2EDuration="1.434895493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:45:01.431788778 +0000 UTC m=+1283.637774176" watchObservedRunningTime="2026-09-08 18:45:01.434895493 +0000 UTC m=+1283.640880890" Sep 08 18:45:02.409506 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:02.409477 2555 generic.go:356] "Generic (PLEG): container finished" podID="da3000a7-1ea1-4009-97e9-62483f7d70db" containerID="a9a5238a98d6168e5b3c4999538e60e9446450d22b61f6ded9ec9e4cc66a23d6" exitCode=0 Sep 08 18:45:02.410038 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:02.409569 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-3b04d6-29814885-vqkhv" event={"ID":"da3000a7-1ea1-4009-97e9-62483f7d70db","Type":"ContainerDied","Data":"a9a5238a98d6168e5b3c4999538e60e9446450d22b61f6ded9ec9e4cc66a23d6"} Sep 08 18:45:02.411211 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:02.411192 2555 generic.go:356] "Generic (PLEG): container finished" podID="33a34f27-be15-4794-9d9d-1250d3df9570" containerID="03737ea2c427e69c29f1aeb1e1f53c2ce7000cb7b77ff59b21ca19711b05c51d" exitCode=0 Sep 08 18:45:02.411297 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:02.411231 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6255f2-29814885-7g47b" event={"ID":"33a34f27-be15-4794-9d9d-1250d3df9570","Type":"ContainerDied","Data":"03737ea2c427e69c29f1aeb1e1f53c2ce7000cb7b77ff59b21ca19711b05c51d"} Sep 08 18:45:02.545091 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:02.545071 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-792c35-29814885-ttblp" Sep 08 18:45:03.418004 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:03.417975 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-792c35-29814885-ttblp" Sep 08 18:45:03.418425 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:03.418014 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-792c35-29814885-ttblp" event={"ID":"f4eff982-06b0-410a-887d-b4d4cb85336e","Type":"ContainerDied","Data":"90b1390e18d796eccaa878b8266d02676f0c0a50be3efce746c9db506c9792ba"} Sep 08 18:45:03.418425 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:03.418049 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="90b1390e18d796eccaa878b8266d02676f0c0a50be3efce746c9db506c9792ba" Sep 08 18:45:03.561151 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:03.561131 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6255f2-29814885-7g47b" Sep 08 18:45:03.581307 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:03.581078 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-3b04d6-29814885-vqkhv" Sep 08 18:45:04.424998 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:04.424978 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6255f2-29814885-7g47b" Sep 08 18:45:04.425394 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:04.424991 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6255f2-29814885-7g47b" event={"ID":"33a34f27-be15-4794-9d9d-1250d3df9570","Type":"ContainerDied","Data":"420a3dd164f3f8e55391263ead324e7b047078b12b1ff6154cac04e8d3beddba"} Sep 08 18:45:04.425394 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:04.425037 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="420a3dd164f3f8e55391263ead324e7b047078b12b1ff6154cac04e8d3beddba" Sep 08 18:45:04.426296 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:04.426281 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-3b04d6-29814885-vqkhv" Sep 08 18:45:04.426418 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:04.426319 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-3b04d6-29814885-vqkhv" event={"ID":"da3000a7-1ea1-4009-97e9-62483f7d70db","Type":"ContainerDied","Data":"a9b3e50223aedaa5d6b238ef5e28381be77014757045ca97ead4046495e443ed"} Sep 08 18:45:04.426418 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:04.426349 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a9b3e50223aedaa5d6b238ef5e28381be77014757045ca97ead4046495e443ed" Sep 08 18:45:11.134092 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:11.134054 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:45:24.669975 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:24.669922 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:45:24.970204 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:24.970133 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:45:27.626843 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:27.626810 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:45:30.653363 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:30.653329 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:45:30.754912 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:30.754876 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:45:30.906382 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:30.906300 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:45:32.098698 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:32.098655 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:45:32.208920 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:32.208889 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:45:32.349696 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:32.349594 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:45:32.903277 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:32.903249 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:45:33.004539 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:33.004513 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:45:33.146837 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:33.146801 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:45:34.325173 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:34.325143 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:45:40.649694 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:40.649659 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:45:40.727822 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:40.727788 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:45:40.797370 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:40.797336 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:45:46.535767 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:46.535733 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:45:46.673394 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:46.673350 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:45:46.740363 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:46.740324 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:45:46.831183 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:46.831152 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:45:46.922330 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:46.922297 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:45:48.169745 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:48.169715 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7578-4ab14d"] Sep 08 18:45:48.170136 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:48.170123 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33a34f27-be15-4794-9d9d-1250d3df9570" containerName="cleanup" Sep 08 18:45:48.170180 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:48.170139 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="33a34f27-be15-4794-9d9d-1250d3df9570" containerName="cleanup" Sep 08 18:45:48.170180 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:48.170160 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da3000a7-1ea1-4009-97e9-62483f7d70db" containerName="cleanup" Sep 08 18:45:48.170180 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:48.170165 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="da3000a7-1ea1-4009-97e9-62483f7d70db" containerName="cleanup" Sep 08 18:45:48.170180 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:48.170173 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4eff982-06b0-410a-887d-b4d4cb85336e" containerName="cleanup" Sep 08 18:45:48.170180 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:48.170177 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4eff982-06b0-410a-887d-b4d4cb85336e" containerName="cleanup" Sep 08 18:45:48.170334 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:48.170236 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="33a34f27-be15-4794-9d9d-1250d3df9570" containerName="cleanup" Sep 08 18:45:48.170334 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:48.170243 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4eff982-06b0-410a-887d-b4d4cb85336e" containerName="cleanup" Sep 08 18:45:48.170334 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:48.170253 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="da3000a7-1ea1-4009-97e9-62483f7d70db" containerName="cleanup" Sep 08 18:45:48.173293 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:48.173278 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7578-4ab14d" Sep 08 18:45:48.175769 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:48.175744 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-mhbkz\"" Sep 08 18:45:48.179590 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:48.179565 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7578-4ab14d"] Sep 08 18:45:48.277453 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:48.277417 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6sdn\" (UniqueName: \"kubernetes.io/projected/b4a6630b-174b-4cb3-9398-655d8d454e58-kube-api-access-k6sdn\") pod \"test-curl-7578-4ab14d\" (UID: \"b4a6630b-174b-4cb3-9398-655d8d454e58\") " pod="opendatahub/test-curl-7578-4ab14d" Sep 08 18:45:48.378275 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:48.378231 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6sdn\" (UniqueName: \"kubernetes.io/projected/b4a6630b-174b-4cb3-9398-655d8d454e58-kube-api-access-k6sdn\") pod \"test-curl-7578-4ab14d\" (UID: \"b4a6630b-174b-4cb3-9398-655d8d454e58\") " pod="opendatahub/test-curl-7578-4ab14d" Sep 08 18:45:48.388482 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:48.388453 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6sdn\" (UniqueName: \"kubernetes.io/projected/b4a6630b-174b-4cb3-9398-655d8d454e58-kube-api-access-k6sdn\") pod \"test-curl-7578-4ab14d\" (UID: \"b4a6630b-174b-4cb3-9398-655d8d454e58\") " pod="opendatahub/test-curl-7578-4ab14d" Sep 08 18:45:48.483843 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:48.483773 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7578-4ab14d" Sep 08 18:45:48.751708 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:48.751666 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7578-4ab14d"] Sep 08 18:45:48.752881 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:45:48.752858 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb4a6630b_174b_4cb3_9398_655d8d454e58.slice/crio-108b4f7c2491e760c3f6f36924e805fdd8198ae6c4432d2d01bd99ab37fc69aa WatchSource:0}: Error finding container 108b4f7c2491e760c3f6f36924e805fdd8198ae6c4432d2d01bd99ab37fc69aa: Status 404 returned error can't find the container with id 108b4f7c2491e760c3f6f36924e805fdd8198ae6c4432d2d01bd99ab37fc69aa Sep 08 18:45:49.690034 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:49.689990 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7578-4ab14d" event={"ID":"b4a6630b-174b-4cb3-9398-655d8d454e58","Type":"ContainerStarted","Data":"631803e96883bb7a8394586f0e57cdf636a13a2b5035a91b5423e57eb7a0ce57"} Sep 08 18:45:49.690410 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:49.690049 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7578-4ab14d" event={"ID":"b4a6630b-174b-4cb3-9398-655d8d454e58","Type":"ContainerStarted","Data":"108b4f7c2491e760c3f6f36924e805fdd8198ae6c4432d2d01bd99ab37fc69aa"} Sep 08 18:45:49.703938 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:49.703899 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7578-4ab14d" podStartSLOduration=1.703882123 podStartE2EDuration="1.703882123s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:45:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:45:49.703657766 +0000 UTC m=+1331.909643165" watchObservedRunningTime="2026-09-08 18:45:49.703882123 +0000 UTC m=+1331.909867521" Sep 08 18:45:50.297353 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:50.297271 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7578-4ab14d"] Sep 08 18:45:50.300186 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:50.300157 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7578-4ab14d"] Sep 08 18:45:50.696529 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:45:50.696501 2555 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7578-4ab14d" container="test-curl-7578-4ab14d" Sep 08 18:45:51.634044 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:51.634009 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:45:51.702375 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:51.702343 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7578-4ab14d" podUID="b4a6630b-174b-4cb3-9398-655d8d454e58" containerName="test-curl-7578-4ab14d" containerID="cri-o://631803e96883bb7a8394586f0e57cdf636a13a2b5035a91b5423e57eb7a0ce57" gracePeriod=2 Sep 08 18:45:52.542198 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:52.542166 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:45:53.876453 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:53.876433 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7578-4ab14d" Sep 08 18:45:53.931246 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:53.931225 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k6sdn\" (UniqueName: \"kubernetes.io/projected/b4a6630b-174b-4cb3-9398-655d8d454e58-kube-api-access-k6sdn\") pod \"b4a6630b-174b-4cb3-9398-655d8d454e58\" (UID: \"b4a6630b-174b-4cb3-9398-655d8d454e58\") " Sep 08 18:45:53.933199 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:53.933173 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4a6630b-174b-4cb3-9398-655d8d454e58-kube-api-access-k6sdn" (OuterVolumeSpecName: "kube-api-access-k6sdn") pod "b4a6630b-174b-4cb3-9398-655d8d454e58" (UID: "b4a6630b-174b-4cb3-9398-655d8d454e58"). InnerVolumeSpecName "kube-api-access-k6sdn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:45:54.032827 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:54.032782 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6sdn\" (UniqueName: \"kubernetes.io/projected/b4a6630b-174b-4cb3-9398-655d8d454e58-kube-api-access-k6sdn\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:45:54.324580 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:54.324554 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4a6630b-174b-4cb3-9398-655d8d454e58" path="/var/lib/kubelet/pods/b4a6630b-174b-4cb3-9398-655d8d454e58/volumes" Sep 08 18:45:54.722163 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:54.722100 2555 generic.go:356] "Generic (PLEG): container finished" podID="b4a6630b-174b-4cb3-9398-655d8d454e58" containerID="631803e96883bb7a8394586f0e57cdf636a13a2b5035a91b5423e57eb7a0ce57" exitCode=137 Sep 08 18:45:54.722163 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:54.722139 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7578-4ab14d" Sep 08 18:45:54.722317 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:54.722195 2555 scope.go:117] "RemoveContainer" containerID="631803e96883bb7a8394586f0e57cdf636a13a2b5035a91b5423e57eb7a0ce57" Sep 08 18:45:54.731500 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:54.731483 2555 scope.go:117] "RemoveContainer" containerID="631803e96883bb7a8394586f0e57cdf636a13a2b5035a91b5423e57eb7a0ce57" Sep 08 18:45:54.731782 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:45:54.731762 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"631803e96883bb7a8394586f0e57cdf636a13a2b5035a91b5423e57eb7a0ce57\": container with ID starting with 631803e96883bb7a8394586f0e57cdf636a13a2b5035a91b5423e57eb7a0ce57 not found: ID does not exist" containerID="631803e96883bb7a8394586f0e57cdf636a13a2b5035a91b5423e57eb7a0ce57" Sep 08 18:45:54.731889 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:54.731789 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"631803e96883bb7a8394586f0e57cdf636a13a2b5035a91b5423e57eb7a0ce57"} err="failed to get container status \"631803e96883bb7a8394586f0e57cdf636a13a2b5035a91b5423e57eb7a0ce57\": rpc error: code = NotFound desc = could not find container \"631803e96883bb7a8394586f0e57cdf636a13a2b5035a91b5423e57eb7a0ce57\": container with ID starting with 631803e96883bb7a8394586f0e57cdf636a13a2b5035a91b5423e57eb7a0ce57 not found: ID does not exist" Sep 08 18:45:57.357381 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:57.357344 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:45:57.588132 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:57.588106 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:45:57.731956 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:45:57.731893 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:46:02.138956 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:02.138867 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:46:07.050790 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:07.050752 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:46:12.368838 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:12.368807 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:46:18.529036 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:18.529006 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:46:21.735237 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:21.735197 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:46:24.556008 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:24.555977 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:46:24.658809 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:24.658776 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:46:24.897858 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:24.897774 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:46:25.180360 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.180286 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb"] Sep 08 18:46:25.180721 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.180704 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4a6630b-174b-4cb3-9398-655d8d454e58" containerName="test-curl-7578-4ab14d" Sep 08 18:46:25.180721 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.180721 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4a6630b-174b-4cb3-9398-655d8d454e58" containerName="test-curl-7578-4ab14d" Sep 08 18:46:25.180887 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.180873 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4a6630b-174b-4cb3-9398-655d8d454e58" containerName="test-curl-7578-4ab14d" Sep 08 18:46:25.184110 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.184095 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:46:25.186411 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.186392 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-ff349c-openshift-default-dockercfg-7rfts\"" Sep 08 18:46:25.194915 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.194897 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fde069a-a7c7-467e-bc1f-ee5820163206-istio-token\") pod \"e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb\" (UID: \"0fde069a-a7c7-467e-bc1f-ee5820163206\") " pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:46:25.195013 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.194930 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fde069a-a7c7-467e-bc1f-ee5820163206-credential-socket\") pod \"e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb\" (UID: \"0fde069a-a7c7-467e-bc1f-ee5820163206\") " pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:46:25.195013 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.194952 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb"] Sep 08 18:46:25.195013 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.195001 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fde069a-a7c7-467e-bc1f-ee5820163206-istio-data\") pod \"e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb\" (UID: \"0fde069a-a7c7-467e-bc1f-ee5820163206\") " pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:46:25.195186 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.195045 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fde069a-a7c7-467e-bc1f-ee5820163206-istiod-ca-cert\") pod \"e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb\" (UID: \"0fde069a-a7c7-467e-bc1f-ee5820163206\") " pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:46:25.195186 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.195079 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fde069a-a7c7-467e-bc1f-ee5820163206-workload-socket\") pod \"e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb\" (UID: \"0fde069a-a7c7-467e-bc1f-ee5820163206\") " pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:46:25.195186 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.195103 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fde069a-a7c7-467e-bc1f-ee5820163206-istio-envoy\") pod \"e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb\" (UID: \"0fde069a-a7c7-467e-bc1f-ee5820163206\") " pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:46:25.195186 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.195169 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fde069a-a7c7-467e-bc1f-ee5820163206-workload-certs\") pod \"e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb\" (UID: \"0fde069a-a7c7-467e-bc1f-ee5820163206\") " pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:46:25.195362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.195197 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5tnw\" (UniqueName: \"kubernetes.io/projected/0fde069a-a7c7-467e-bc1f-ee5820163206-kube-api-access-r5tnw\") pod \"e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb\" (UID: \"0fde069a-a7c7-467e-bc1f-ee5820163206\") " pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:46:25.195362 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.195280 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fde069a-a7c7-467e-bc1f-ee5820163206-istio-podinfo\") pod \"e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb\" (UID: \"0fde069a-a7c7-467e-bc1f-ee5820163206\") " pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:46:25.203877 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.203853 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:46:25.296354 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.296329 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fde069a-a7c7-467e-bc1f-ee5820163206-istio-token\") pod \"e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb\" (UID: \"0fde069a-a7c7-467e-bc1f-ee5820163206\") " pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:46:25.296464 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.296366 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fde069a-a7c7-467e-bc1f-ee5820163206-credential-socket\") pod \"e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb\" (UID: \"0fde069a-a7c7-467e-bc1f-ee5820163206\") " pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:46:25.296464 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.296415 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fde069a-a7c7-467e-bc1f-ee5820163206-istio-data\") pod \"e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb\" (UID: \"0fde069a-a7c7-467e-bc1f-ee5820163206\") " pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:46:25.296464 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.296433 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fde069a-a7c7-467e-bc1f-ee5820163206-istiod-ca-cert\") pod \"e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb\" (UID: \"0fde069a-a7c7-467e-bc1f-ee5820163206\") " pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:46:25.296464 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.296451 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fde069a-a7c7-467e-bc1f-ee5820163206-workload-socket\") pod \"e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb\" (UID: \"0fde069a-a7c7-467e-bc1f-ee5820163206\") " pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:46:25.296656 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.296472 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fde069a-a7c7-467e-bc1f-ee5820163206-istio-envoy\") pod \"e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb\" (UID: \"0fde069a-a7c7-467e-bc1f-ee5820163206\") " pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:46:25.296656 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.296518 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fde069a-a7c7-467e-bc1f-ee5820163206-workload-certs\") pod \"e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb\" (UID: \"0fde069a-a7c7-467e-bc1f-ee5820163206\") " pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:46:25.296656 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.296544 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5tnw\" (UniqueName: \"kubernetes.io/projected/0fde069a-a7c7-467e-bc1f-ee5820163206-kube-api-access-r5tnw\") pod \"e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb\" (UID: \"0fde069a-a7c7-467e-bc1f-ee5820163206\") " pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:46:25.296656 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.296607 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fde069a-a7c7-467e-bc1f-ee5820163206-istio-podinfo\") pod \"e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb\" (UID: \"0fde069a-a7c7-467e-bc1f-ee5820163206\") " pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:46:25.296893 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.296872 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fde069a-a7c7-467e-bc1f-ee5820163206-istio-data\") pod \"e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb\" (UID: \"0fde069a-a7c7-467e-bc1f-ee5820163206\") " pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:46:25.296998 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.296975 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fde069a-a7c7-467e-bc1f-ee5820163206-workload-socket\") pod \"e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb\" (UID: \"0fde069a-a7c7-467e-bc1f-ee5820163206\") " pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:46:25.297062 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.297001 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fde069a-a7c7-467e-bc1f-ee5820163206-workload-certs\") pod \"e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb\" (UID: \"0fde069a-a7c7-467e-bc1f-ee5820163206\") " pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:46:25.297117 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.297054 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fde069a-a7c7-467e-bc1f-ee5820163206-credential-socket\") pod \"e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb\" (UID: \"0fde069a-a7c7-467e-bc1f-ee5820163206\") " pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:46:25.297311 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.297291 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fde069a-a7c7-467e-bc1f-ee5820163206-istiod-ca-cert\") pod \"e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb\" (UID: \"0fde069a-a7c7-467e-bc1f-ee5820163206\") " pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:46:25.298850 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.298825 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fde069a-a7c7-467e-bc1f-ee5820163206-istio-envoy\") pod \"e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb\" (UID: \"0fde069a-a7c7-467e-bc1f-ee5820163206\") " pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:46:25.299248 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.299228 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fde069a-a7c7-467e-bc1f-ee5820163206-istio-podinfo\") pod \"e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb\" (UID: \"0fde069a-a7c7-467e-bc1f-ee5820163206\") " pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:46:25.304001 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.303977 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fde069a-a7c7-467e-bc1f-ee5820163206-istio-token\") pod \"e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb\" (UID: \"0fde069a-a7c7-467e-bc1f-ee5820163206\") " pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:46:25.304158 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.304139 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5tnw\" (UniqueName: \"kubernetes.io/projected/0fde069a-a7c7-467e-bc1f-ee5820163206-kube-api-access-r5tnw\") pod \"e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb\" (UID: \"0fde069a-a7c7-467e-bc1f-ee5820163206\") " pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:46:25.494756 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.494678 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:46:25.654707 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.654654 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:46:25.773058 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.773028 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb"] Sep 08 18:46:25.774172 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:46:25.774148 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fde069a_a7c7_467e_bc1f_ee5820163206.slice/crio-fd65e9afa49dfe22cfc617e4c18ebca576fa9fd3e082057bf4eb6e75eea5d5d4 WatchSource:0}: Error finding container fd65e9afa49dfe22cfc617e4c18ebca576fa9fd3e082057bf4eb6e75eea5d5d4: Status 404 returned error can't find the container with id fd65e9afa49dfe22cfc617e4c18ebca576fa9fd3e082057bf4eb6e75eea5d5d4 Sep 08 18:46:25.776349 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.776320 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 18:46:25.776415 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.776396 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 18:46:25.776458 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.776423 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 18:46:25.909501 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.909461 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" event={"ID":"0fde069a-a7c7-467e-bc1f-ee5820163206","Type":"ContainerStarted","Data":"85ae468f63338512e70158f043b275aa4656084103b76a33b0a12f51fad5ad83"} Sep 08 18:46:25.909666 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.909533 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" event={"ID":"0fde069a-a7c7-467e-bc1f-ee5820163206","Type":"ContainerStarted","Data":"fd65e9afa49dfe22cfc617e4c18ebca576fa9fd3e082057bf4eb6e75eea5d5d4"} Sep 08 18:46:25.932375 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:25.932330 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" podStartSLOduration=0.932314455 podStartE2EDuration="932.314455ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:46:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:46:25.930432335 +0000 UTC m=+1368.136417759" watchObservedRunningTime="2026-09-08 18:46:25.932314455 +0000 UTC m=+1368.138299850" Sep 08 18:46:26.164842 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:26.164809 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:46:26.495744 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:26.495631 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:46:26.501867 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:26.501842 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:46:26.915317 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:26.915291 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:46:26.916380 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:26.916357 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:46:27.862395 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:27.862360 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:46:27.970894 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:27.970857 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:46:29.171705 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:29.171655 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:46:29.247229 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:29.247196 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:46:29.785461 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:29.785422 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ff349c-5cf6d89bcc-7rxzr"] Sep 08 18:46:29.789280 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:29.789256 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ff349c-5cf6d89bcc-7rxzr" Sep 08 18:46:29.791509 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:29.791489 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-ff349c\"" Sep 08 18:46:29.791634 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:29.791522 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-ff349c-dockercfg-mj7rp\"" Sep 08 18:46:29.797961 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:29.797939 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ff349c-5cf6d89bcc-7rxzr"] Sep 08 18:46:29.839389 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:29.839358 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhcx9\" (UniqueName: \"kubernetes.io/projected/c56b5fa2-71aa-4315-bc0a-6d4f24c9067f-kube-api-access-hhcx9\") pod \"payload-processing-e2e-api-a-ff349c-5cf6d89bcc-7rxzr\" (UID: \"c56b5fa2-71aa-4315-bc0a-6d4f24c9067f\") " pod="openshift-ingress/payload-processing-e2e-api-a-ff349c-5cf6d89bcc-7rxzr" Sep 08 18:46:29.839564 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:29.839409 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c56b5fa2-71aa-4315-bc0a-6d4f24c9067f-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ff349c-5cf6d89bcc-7rxzr\" (UID: \"c56b5fa2-71aa-4315-bc0a-6d4f24c9067f\") " pod="openshift-ingress/payload-processing-e2e-api-a-ff349c-5cf6d89bcc-7rxzr" Sep 08 18:46:29.941012 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:29.940982 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhcx9\" (UniqueName: \"kubernetes.io/projected/c56b5fa2-71aa-4315-bc0a-6d4f24c9067f-kube-api-access-hhcx9\") pod \"payload-processing-e2e-api-a-ff349c-5cf6d89bcc-7rxzr\" (UID: \"c56b5fa2-71aa-4315-bc0a-6d4f24c9067f\") " pod="openshift-ingress/payload-processing-e2e-api-a-ff349c-5cf6d89bcc-7rxzr" Sep 08 18:46:29.941189 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:29.941046 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c56b5fa2-71aa-4315-bc0a-6d4f24c9067f-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ff349c-5cf6d89bcc-7rxzr\" (UID: \"c56b5fa2-71aa-4315-bc0a-6d4f24c9067f\") " pod="openshift-ingress/payload-processing-e2e-api-a-ff349c-5cf6d89bcc-7rxzr" Sep 08 18:46:29.941707 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:29.941674 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c56b5fa2-71aa-4315-bc0a-6d4f24c9067f-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ff349c-5cf6d89bcc-7rxzr\" (UID: \"c56b5fa2-71aa-4315-bc0a-6d4f24c9067f\") " pod="openshift-ingress/payload-processing-e2e-api-a-ff349c-5cf6d89bcc-7rxzr" Sep 08 18:46:29.948757 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:29.948737 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhcx9\" (UniqueName: \"kubernetes.io/projected/c56b5fa2-71aa-4315-bc0a-6d4f24c9067f-kube-api-access-hhcx9\") pod \"payload-processing-e2e-api-a-ff349c-5cf6d89bcc-7rxzr\" (UID: \"c56b5fa2-71aa-4315-bc0a-6d4f24c9067f\") " pod="openshift-ingress/payload-processing-e2e-api-a-ff349c-5cf6d89bcc-7rxzr" Sep 08 18:46:30.061447 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:30.061401 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:46:30.101717 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:30.101663 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ff349c-5cf6d89bcc-7rxzr" Sep 08 18:46:30.166171 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:30.166101 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:46:30.376968 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:30.376938 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ff349c-5cf6d89bcc-7rxzr"] Sep 08 18:46:30.378142 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:46:30.378109 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc56b5fa2_71aa_4315_bc0a_6d4f24c9067f.slice/crio-284f8f766638625a6788b711540be40f150f8d0d7170ffd509e690149c693659 WatchSource:0}: Error finding container 284f8f766638625a6788b711540be40f150f8d0d7170ffd509e690149c693659: Status 404 returned error can't find the container with id 284f8f766638625a6788b711540be40f150f8d0d7170ffd509e690149c693659 Sep 08 18:46:30.939363 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:30.939323 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ff349c-5cf6d89bcc-7rxzr" event={"ID":"c56b5fa2-71aa-4315-bc0a-6d4f24c9067f","Type":"ContainerStarted","Data":"bad72ff4e3bf3e94a3e007a7d7e9ef45d08962f129308883960b7aa6d17290b8"} Sep 08 18:46:30.939574 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:30.939558 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-ff349c-5cf6d89bcc-7rxzr" Sep 08 18:46:30.939642 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:30.939599 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ff349c-5cf6d89bcc-7rxzr" event={"ID":"c56b5fa2-71aa-4315-bc0a-6d4f24c9067f","Type":"ContainerStarted","Data":"284f8f766638625a6788b711540be40f150f8d0d7170ffd509e690149c693659"} Sep 08 18:46:30.959254 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:30.959204 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-ff349c-5cf6d89bcc-7rxzr" podStartSLOduration=1.959188738 podStartE2EDuration="1.959188738s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:46:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:46:30.956525371 +0000 UTC m=+1373.162510767" watchObservedRunningTime="2026-09-08 18:46:30.959188738 +0000 UTC m=+1373.165174134" Sep 08 18:46:41.947304 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:41.947269 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-ff349c-5cf6d89bcc-7rxzr" Sep 08 18:46:42.230648 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:42.230547 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:46:46.043181 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:46.043145 2555 generic.go:356] "Generic (PLEG): container finished" podID="0fde069a-a7c7-467e-bc1f-ee5820163206" containerID="85ae468f63338512e70158f043b275aa4656084103b76a33b0a12f51fad5ad83" exitCode=0 Sep 08 18:46:46.043600 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:46.043219 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" event={"ID":"0fde069a-a7c7-467e-bc1f-ee5820163206","Type":"ContainerDied","Data":"85ae468f63338512e70158f043b275aa4656084103b76a33b0a12f51fad5ad83"} Sep 08 18:46:46.043666 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:46.043652 2555 scope.go:117] "RemoveContainer" containerID="85ae468f63338512e70158f043b275aa4656084103b76a33b0a12f51fad5ad83" Sep 08 18:46:46.044903 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:46.044879 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 18:46:46.044960 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:46.044945 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 18:46:46.044996 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:46.044976 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 18:46:47.051581 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:47.051547 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" event={"ID":"0fde069a-a7c7-467e-bc1f-ee5820163206","Type":"ContainerStarted","Data":"420e69178ebde9fbd8e254be192126e2874d25f87f85265327f9c93a80907120"} Sep 08 18:46:47.495410 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:47.495304 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:46:47.495725 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:47.495399 2555 patch_prober.go:28] interesting pod/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Sep 08 18:46:47.495911 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:47.495880 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" podUID="0fde069a-a7c7-467e-bc1f-ee5820163206" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Sep 08 18:46:48.496228 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:48.496169 2555 patch_prober.go:28] interesting pod/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Sep 08 18:46:48.496875 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:48.496841 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" podUID="0fde069a-a7c7-467e-bc1f-ee5820163206" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Sep 08 18:46:49.509956 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:49.509911 2555 patch_prober.go:28] interesting pod/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Sep 08 18:46:49.510554 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:49.510520 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" podUID="0fde069a-a7c7-467e-bc1f-ee5820163206" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Sep 08 18:46:50.496258 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:50.496206 2555 patch_prober.go:28] interesting pod/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Sep 08 18:46:50.496487 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:50.496284 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" podUID="0fde069a-a7c7-467e-bc1f-ee5820163206" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Sep 08 18:46:51.496187 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:51.496134 2555 patch_prober.go:28] interesting pod/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Sep 08 18:46:51.496736 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:51.496228 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" podUID="0fde069a-a7c7-467e-bc1f-ee5820163206" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Sep 08 18:46:52.565511 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:52.565471 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:46:52.565983 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:52.565805 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:46:52.566479 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:52.566464 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:46:52.846519 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:52.846427 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:46:53.293313 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:53.293282 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:46:53.601408 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:53.601303 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:46:53.809552 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:53.809507 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:46:56.943824 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:56.943786 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:46:57.042977 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:57.042895 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:46:58.106172 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:58.106135 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:46:58.206965 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:58.206915 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:46:58.307086 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:58.307054 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:46:58.578893 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:58.578857 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:46:58.659646 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:58.659607 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:46:58.692793 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:58.692753 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-179a6f-68c5f896cd-wg7q2"] Sep 08 18:46:58.696622 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:58.696603 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-179a6f-68c5f896cd-wg7q2" Sep 08 18:46:58.698596 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:58.698571 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-179a6f\"" Sep 08 18:46:58.698767 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:58.698640 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-179a6f-dockercfg-t4k8h\"" Sep 08 18:46:58.705636 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:58.705603 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-179a6f-68c5f896cd-wg7q2"] Sep 08 18:46:58.715164 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:58.715123 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-179a6f-575f9bc89c-ggs9b"] Sep 08 18:46:58.719119 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:58.719094 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-179a6f-575f9bc89c-ggs9b" Sep 08 18:46:58.729866 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:58.729838 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-179a6f-575f9bc89c-ggs9b"] Sep 08 18:46:58.837349 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:58.837250 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbtq4\" (UniqueName: \"kubernetes.io/projected/d507f7eb-bb08-4147-924d-f02563c7d445-kube-api-access-hbtq4\") pod \"payload-processing-e2e-api-b-179a6f-575f9bc89c-ggs9b\" (UID: \"d507f7eb-bb08-4147-924d-f02563c7d445\") " pod="openshift-ingress/payload-processing-e2e-api-b-179a6f-575f9bc89c-ggs9b" Sep 08 18:46:58.837349 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:58.837304 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/788a9706-0aa4-4918-8c84-43e372f7db05-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-179a6f-68c5f896cd-wg7q2\" (UID: \"788a9706-0aa4-4918-8c84-43e372f7db05\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-179a6f-68c5f896cd-wg7q2" Sep 08 18:46:58.837586 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:58.837361 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxbhf\" (UniqueName: \"kubernetes.io/projected/788a9706-0aa4-4918-8c84-43e372f7db05-kube-api-access-zxbhf\") pod \"payload-pre-processing-e2e-api-b-179a6f-68c5f896cd-wg7q2\" (UID: \"788a9706-0aa4-4918-8c84-43e372f7db05\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-179a6f-68c5f896cd-wg7q2" Sep 08 18:46:58.837586 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:58.837478 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d507f7eb-bb08-4147-924d-f02563c7d445-plugins-config-volume\") pod \"payload-processing-e2e-api-b-179a6f-575f9bc89c-ggs9b\" (UID: \"d507f7eb-bb08-4147-924d-f02563c7d445\") " pod="openshift-ingress/payload-processing-e2e-api-b-179a6f-575f9bc89c-ggs9b" Sep 08 18:46:58.939084 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:58.939048 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d507f7eb-bb08-4147-924d-f02563c7d445-plugins-config-volume\") pod \"payload-processing-e2e-api-b-179a6f-575f9bc89c-ggs9b\" (UID: \"d507f7eb-bb08-4147-924d-f02563c7d445\") " pod="openshift-ingress/payload-processing-e2e-api-b-179a6f-575f9bc89c-ggs9b" Sep 08 18:46:58.939289 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:58.939093 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbtq4\" (UniqueName: \"kubernetes.io/projected/d507f7eb-bb08-4147-924d-f02563c7d445-kube-api-access-hbtq4\") pod \"payload-processing-e2e-api-b-179a6f-575f9bc89c-ggs9b\" (UID: \"d507f7eb-bb08-4147-924d-f02563c7d445\") " pod="openshift-ingress/payload-processing-e2e-api-b-179a6f-575f9bc89c-ggs9b" Sep 08 18:46:58.939289 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:58.939136 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/788a9706-0aa4-4918-8c84-43e372f7db05-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-179a6f-68c5f896cd-wg7q2\" (UID: \"788a9706-0aa4-4918-8c84-43e372f7db05\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-179a6f-68c5f896cd-wg7q2" Sep 08 18:46:58.939289 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:58.939173 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxbhf\" (UniqueName: \"kubernetes.io/projected/788a9706-0aa4-4918-8c84-43e372f7db05-kube-api-access-zxbhf\") pod \"payload-pre-processing-e2e-api-b-179a6f-68c5f896cd-wg7q2\" (UID: \"788a9706-0aa4-4918-8c84-43e372f7db05\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-179a6f-68c5f896cd-wg7q2" Sep 08 18:46:58.939877 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:58.939859 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d507f7eb-bb08-4147-924d-f02563c7d445-plugins-config-volume\") pod \"payload-processing-e2e-api-b-179a6f-575f9bc89c-ggs9b\" (UID: \"d507f7eb-bb08-4147-924d-f02563c7d445\") " pod="openshift-ingress/payload-processing-e2e-api-b-179a6f-575f9bc89c-ggs9b" Sep 08 18:46:58.939937 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:58.939915 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/788a9706-0aa4-4918-8c84-43e372f7db05-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-179a6f-68c5f896cd-wg7q2\" (UID: \"788a9706-0aa4-4918-8c84-43e372f7db05\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-179a6f-68c5f896cd-wg7q2" Sep 08 18:46:58.947613 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:58.947588 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxbhf\" (UniqueName: \"kubernetes.io/projected/788a9706-0aa4-4918-8c84-43e372f7db05-kube-api-access-zxbhf\") pod \"payload-pre-processing-e2e-api-b-179a6f-68c5f896cd-wg7q2\" (UID: \"788a9706-0aa4-4918-8c84-43e372f7db05\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-179a6f-68c5f896cd-wg7q2" Sep 08 18:46:58.947773 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:58.947594 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbtq4\" (UniqueName: \"kubernetes.io/projected/d507f7eb-bb08-4147-924d-f02563c7d445-kube-api-access-hbtq4\") pod \"payload-processing-e2e-api-b-179a6f-575f9bc89c-ggs9b\" (UID: \"d507f7eb-bb08-4147-924d-f02563c7d445\") " pod="openshift-ingress/payload-processing-e2e-api-b-179a6f-575f9bc89c-ggs9b" Sep 08 18:46:59.011399 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:59.011361 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-179a6f-68c5f896cd-wg7q2" Sep 08 18:46:59.032832 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:59.032805 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-179a6f-575f9bc89c-ggs9b" Sep 08 18:46:59.290305 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:59.290277 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-179a6f-68c5f896cd-wg7q2"] Sep 08 18:46:59.291276 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:46:59.291251 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod788a9706_0aa4_4918_8c84_43e372f7db05.slice/crio-52ab5657d7703267be0e151fd7c11ddf42ed38e6d9463d8ec6f3c394c09dce73 WatchSource:0}: Error finding container 52ab5657d7703267be0e151fd7c11ddf42ed38e6d9463d8ec6f3c394c09dce73: Status 404 returned error can't find the container with id 52ab5657d7703267be0e151fd7c11ddf42ed38e6d9463d8ec6f3c394c09dce73 Sep 08 18:46:59.320281 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:46:59.320258 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-179a6f-575f9bc89c-ggs9b"] Sep 08 18:46:59.324929 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:46:59.324905 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd507f7eb_bb08_4147_924d_f02563c7d445.slice/crio-139b647f902c01a3bbafac39325b4b83ce920ebe49290d992ca0375463a87793 WatchSource:0}: Error finding container 139b647f902c01a3bbafac39325b4b83ce920ebe49290d992ca0375463a87793: Status 404 returned error can't find the container with id 139b647f902c01a3bbafac39325b4b83ce920ebe49290d992ca0375463a87793 Sep 08 18:47:00.136865 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:00.136826 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-179a6f-575f9bc89c-ggs9b" event={"ID":"d507f7eb-bb08-4147-924d-f02563c7d445","Type":"ContainerStarted","Data":"72f1cf688ab10223d9abc72fec9cd820aebf743b2989964328a97d8c2fa1acde"} Sep 08 18:47:00.137090 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:00.136895 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-179a6f-575f9bc89c-ggs9b" Sep 08 18:47:00.137090 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:00.136920 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-179a6f-575f9bc89c-ggs9b" event={"ID":"d507f7eb-bb08-4147-924d-f02563c7d445","Type":"ContainerStarted","Data":"139b647f902c01a3bbafac39325b4b83ce920ebe49290d992ca0375463a87793"} Sep 08 18:47:00.138588 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:00.138562 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-179a6f-68c5f896cd-wg7q2" event={"ID":"788a9706-0aa4-4918-8c84-43e372f7db05","Type":"ContainerStarted","Data":"161c3361a4cf1f8b5d2f1f8a84573ef28a1feac110b732a450825954553aa796"} Sep 08 18:47:00.138722 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:00.138612 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-179a6f-68c5f896cd-wg7q2" event={"ID":"788a9706-0aa4-4918-8c84-43e372f7db05","Type":"ContainerStarted","Data":"52ab5657d7703267be0e151fd7c11ddf42ed38e6d9463d8ec6f3c394c09dce73"} Sep 08 18:47:00.138722 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:00.138657 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-179a6f-68c5f896cd-wg7q2" Sep 08 18:47:00.160630 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:00.160583 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-179a6f-575f9bc89c-ggs9b" podStartSLOduration=2.160568253 podStartE2EDuration="2.160568253s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:46:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:47:00.159503026 +0000 UTC m=+1402.365488427" watchObservedRunningTime="2026-09-08 18:47:00.160568253 +0000 UTC m=+1402.366553651" Sep 08 18:47:00.176257 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:00.176212 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-179a6f-68c5f896cd-wg7q2" podStartSLOduration=2.176198843 podStartE2EDuration="2.176198843s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:46:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:47:00.174018201 +0000 UTC m=+1402.380003599" watchObservedRunningTime="2026-09-08 18:47:00.176198843 +0000 UTC m=+1402.382184241" Sep 08 18:47:09.524300 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:09.524252 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:47:11.147026 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:11.146987 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-179a6f-575f9bc89c-ggs9b" Sep 08 18:47:11.147524 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:11.147072 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-179a6f-68c5f896cd-wg7q2" Sep 08 18:47:22.386401 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:22.386363 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:47:22.440844 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:22.440808 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:47:24.856045 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:24.856003 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:47:26.822467 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:26.822431 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:47:35.924386 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:35.924300 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:47:36.816413 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:36.816376 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:47:38.730617 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:38.730583 2555 scope.go:117] "RemoveContainer" containerID="27bdb54df60cc0cb0f376acf2a02222ee56735481331210974c1106302c0fce2" Sep 08 18:47:40.305960 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:40.305923 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:47:40.927812 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:40.927772 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:47:40.947093 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:40.947062 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ff349c-5cf6d89bcc-7rxzr"] Sep 08 18:47:40.947515 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:40.947458 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-ff349c-5cf6d89bcc-7rxzr" podUID="c56b5fa2-71aa-4315-bc0a-6d4f24c9067f" containerName="payload-processing" containerID="cri-o://bad72ff4e3bf3e94a3e007a7d7e9ef45d08962f129308883960b7aa6d17290b8" gracePeriod=30 Sep 08 18:47:41.025786 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:41.025756 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:47:41.100107 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:41.100073 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:47:41.160765 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:41.160732 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:47:41.227427 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:41.227393 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ff349c-5cf6d89bcc-7rxzr" Sep 08 18:47:41.250882 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:41.250851 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhcx9\" (UniqueName: \"kubernetes.io/projected/c56b5fa2-71aa-4315-bc0a-6d4f24c9067f-kube-api-access-hhcx9\") pod \"c56b5fa2-71aa-4315-bc0a-6d4f24c9067f\" (UID: \"c56b5fa2-71aa-4315-bc0a-6d4f24c9067f\") " Sep 08 18:47:41.251073 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:41.250927 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c56b5fa2-71aa-4315-bc0a-6d4f24c9067f-plugins-config-volume\") pod \"c56b5fa2-71aa-4315-bc0a-6d4f24c9067f\" (UID: \"c56b5fa2-71aa-4315-bc0a-6d4f24c9067f\") " Sep 08 18:47:41.251401 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:41.251357 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c56b5fa2-71aa-4315-bc0a-6d4f24c9067f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c56b5fa2-71aa-4315-bc0a-6d4f24c9067f" (UID: "c56b5fa2-71aa-4315-bc0a-6d4f24c9067f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:47:41.253862 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:41.253833 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c56b5fa2-71aa-4315-bc0a-6d4f24c9067f-kube-api-access-hhcx9" (OuterVolumeSpecName: "kube-api-access-hhcx9") pod "c56b5fa2-71aa-4315-bc0a-6d4f24c9067f" (UID: "c56b5fa2-71aa-4315-bc0a-6d4f24c9067f"). InnerVolumeSpecName "kube-api-access-hhcx9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:47:41.351849 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:41.351815 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c56b5fa2-71aa-4315-bc0a-6d4f24c9067f-plugins-config-volume\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:47:41.351849 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:41.351843 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhcx9\" (UniqueName: \"kubernetes.io/projected/c56b5fa2-71aa-4315-bc0a-6d4f24c9067f-kube-api-access-hhcx9\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:47:41.411675 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:41.411641 2555 generic.go:356] "Generic (PLEG): container finished" podID="c56b5fa2-71aa-4315-bc0a-6d4f24c9067f" containerID="bad72ff4e3bf3e94a3e007a7d7e9ef45d08962f129308883960b7aa6d17290b8" exitCode=0 Sep 08 18:47:41.411675 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:41.411700 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ff349c-5cf6d89bcc-7rxzr" Sep 08 18:47:41.411939 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:41.411726 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ff349c-5cf6d89bcc-7rxzr" event={"ID":"c56b5fa2-71aa-4315-bc0a-6d4f24c9067f","Type":"ContainerDied","Data":"bad72ff4e3bf3e94a3e007a7d7e9ef45d08962f129308883960b7aa6d17290b8"} Sep 08 18:47:41.411939 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:41.411784 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ff349c-5cf6d89bcc-7rxzr" event={"ID":"c56b5fa2-71aa-4315-bc0a-6d4f24c9067f","Type":"ContainerDied","Data":"284f8f766638625a6788b711540be40f150f8d0d7170ffd509e690149c693659"} Sep 08 18:47:41.411939 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:41.411808 2555 scope.go:117] "RemoveContainer" containerID="bad72ff4e3bf3e94a3e007a7d7e9ef45d08962f129308883960b7aa6d17290b8" Sep 08 18:47:41.422011 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:41.421994 2555 scope.go:117] "RemoveContainer" containerID="bad72ff4e3bf3e94a3e007a7d7e9ef45d08962f129308883960b7aa6d17290b8" Sep 08 18:47:41.422296 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:47:41.422267 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bad72ff4e3bf3e94a3e007a7d7e9ef45d08962f129308883960b7aa6d17290b8\": container with ID starting with bad72ff4e3bf3e94a3e007a7d7e9ef45d08962f129308883960b7aa6d17290b8 not found: ID does not exist" containerID="bad72ff4e3bf3e94a3e007a7d7e9ef45d08962f129308883960b7aa6d17290b8" Sep 08 18:47:41.422365 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:41.422292 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bad72ff4e3bf3e94a3e007a7d7e9ef45d08962f129308883960b7aa6d17290b8"} err="failed to get container status \"bad72ff4e3bf3e94a3e007a7d7e9ef45d08962f129308883960b7aa6d17290b8\": rpc error: code = NotFound desc = could not find container \"bad72ff4e3bf3e94a3e007a7d7e9ef45d08962f129308883960b7aa6d17290b8\": container with ID starting with bad72ff4e3bf3e94a3e007a7d7e9ef45d08962f129308883960b7aa6d17290b8 not found: ID does not exist" Sep 08 18:47:41.435098 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:41.435067 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ff349c-5cf6d89bcc-7rxzr"] Sep 08 18:47:41.440651 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:41.440627 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ff349c-5cf6d89bcc-7rxzr"] Sep 08 18:47:41.441811 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:41.441791 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:47:42.325956 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:42.325919 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c56b5fa2-71aa-4315-bc0a-6d4f24c9067f" path="/var/lib/kubelet/pods/c56b5fa2-71aa-4315-bc0a-6d4f24c9067f/volumes" Sep 08 18:47:44.012744 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:44.012714 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:47:47.236669 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:47.236632 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:47:47.297351 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:47.297316 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:47:47.475625 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:47.475593 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb"] Sep 08 18:47:47.475921 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:47.475895 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" podUID="0fde069a-a7c7-467e-bc1f-ee5820163206" containerName="istio-proxy" containerID="cri-o://420e69178ebde9fbd8e254be192126e2874d25f87f85265327f9c93a80907120" gracePeriod=30 Sep 08 18:47:47.495482 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:47.495395 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:47:48.228016 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:48.227966 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:47:48.230224 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:48.230200 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:47:50.180850 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:50.180810 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:47:52.568189 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:52.568153 2555 patch_prober.go:28] interesting pod/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Sep 08 18:47:52.568605 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:52.568217 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" podUID="0fde069a-a7c7-467e-bc1f-ee5820163206" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Sep 08 18:47:52.732347 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:52.732316 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:47:52.868453 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:52.868416 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fde069a-a7c7-467e-bc1f-ee5820163206-istio-podinfo\") pod \"0fde069a-a7c7-467e-bc1f-ee5820163206\" (UID: \"0fde069a-a7c7-467e-bc1f-ee5820163206\") " Sep 08 18:47:52.868661 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:52.868464 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fde069a-a7c7-467e-bc1f-ee5820163206-workload-certs\") pod \"0fde069a-a7c7-467e-bc1f-ee5820163206\" (UID: \"0fde069a-a7c7-467e-bc1f-ee5820163206\") " Sep 08 18:47:52.868661 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:52.868488 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fde069a-a7c7-467e-bc1f-ee5820163206-istiod-ca-cert\") pod \"0fde069a-a7c7-467e-bc1f-ee5820163206\" (UID: \"0fde069a-a7c7-467e-bc1f-ee5820163206\") " Sep 08 18:47:52.868661 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:52.868516 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fde069a-a7c7-467e-bc1f-ee5820163206-istio-data\") pod \"0fde069a-a7c7-467e-bc1f-ee5820163206\" (UID: \"0fde069a-a7c7-467e-bc1f-ee5820163206\") " Sep 08 18:47:52.868661 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:52.868649 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fde069a-a7c7-467e-bc1f-ee5820163206-istio-envoy\") pod \"0fde069a-a7c7-467e-bc1f-ee5820163206\" (UID: \"0fde069a-a7c7-467e-bc1f-ee5820163206\") " Sep 08 18:47:52.868919 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:52.868722 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5tnw\" (UniqueName: \"kubernetes.io/projected/0fde069a-a7c7-467e-bc1f-ee5820163206-kube-api-access-r5tnw\") pod \"0fde069a-a7c7-467e-bc1f-ee5820163206\" (UID: \"0fde069a-a7c7-467e-bc1f-ee5820163206\") " Sep 08 18:47:52.868919 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:52.868758 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fde069a-a7c7-467e-bc1f-ee5820163206-istio-token\") pod \"0fde069a-a7c7-467e-bc1f-ee5820163206\" (UID: \"0fde069a-a7c7-467e-bc1f-ee5820163206\") " Sep 08 18:47:52.868919 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:52.868772 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fde069a-a7c7-467e-bc1f-ee5820163206-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0fde069a-a7c7-467e-bc1f-ee5820163206" (UID: "0fde069a-a7c7-467e-bc1f-ee5820163206"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:47:52.868919 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:52.868788 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fde069a-a7c7-467e-bc1f-ee5820163206-credential-socket\") pod \"0fde069a-a7c7-467e-bc1f-ee5820163206\" (UID: \"0fde069a-a7c7-467e-bc1f-ee5820163206\") " Sep 08 18:47:52.868919 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:52.868843 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fde069a-a7c7-467e-bc1f-ee5820163206-workload-socket\") pod \"0fde069a-a7c7-467e-bc1f-ee5820163206\" (UID: \"0fde069a-a7c7-467e-bc1f-ee5820163206\") " Sep 08 18:47:52.868919 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:52.868871 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fde069a-a7c7-467e-bc1f-ee5820163206-istio-data" (OuterVolumeSpecName: "istio-data") pod "0fde069a-a7c7-467e-bc1f-ee5820163206" (UID: "0fde069a-a7c7-467e-bc1f-ee5820163206"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:47:52.869225 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:52.869092 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fde069a-a7c7-467e-bc1f-ee5820163206-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0fde069a-a7c7-467e-bc1f-ee5820163206" (UID: "0fde069a-a7c7-467e-bc1f-ee5820163206"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:47:52.869225 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:52.869159 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0fde069a-a7c7-467e-bc1f-ee5820163206-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0fde069a-a7c7-467e-bc1f-ee5820163206" (UID: "0fde069a-a7c7-467e-bc1f-ee5820163206"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:47:52.869371 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:52.869349 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fde069a-a7c7-467e-bc1f-ee5820163206-workload-certs\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:47:52.869551 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:52.869381 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fde069a-a7c7-467e-bc1f-ee5820163206-istiod-ca-cert\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:47:52.869551 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:52.869394 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fde069a-a7c7-467e-bc1f-ee5820163206-istio-data\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:47:52.869551 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:52.869408 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fde069a-a7c7-467e-bc1f-ee5820163206-credential-socket\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:47:52.869551 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:52.869349 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fde069a-a7c7-467e-bc1f-ee5820163206-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0fde069a-a7c7-467e-bc1f-ee5820163206" (UID: "0fde069a-a7c7-467e-bc1f-ee5820163206"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:47:52.871206 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:52.871167 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fde069a-a7c7-467e-bc1f-ee5820163206-kube-api-access-r5tnw" (OuterVolumeSpecName: "kube-api-access-r5tnw") pod "0fde069a-a7c7-467e-bc1f-ee5820163206" (UID: "0fde069a-a7c7-467e-bc1f-ee5820163206"). InnerVolumeSpecName "kube-api-access-r5tnw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:47:52.871315 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:52.871215 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0fde069a-a7c7-467e-bc1f-ee5820163206-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0fde069a-a7c7-467e-bc1f-ee5820163206" (UID: "0fde069a-a7c7-467e-bc1f-ee5820163206"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 08 18:47:52.871536 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:52.871508 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fde069a-a7c7-467e-bc1f-ee5820163206-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0fde069a-a7c7-467e-bc1f-ee5820163206" (UID: "0fde069a-a7c7-467e-bc1f-ee5820163206"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:47:52.871635 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:52.871510 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fde069a-a7c7-467e-bc1f-ee5820163206-istio-token" (OuterVolumeSpecName: "istio-token") pod "0fde069a-a7c7-467e-bc1f-ee5820163206" (UID: "0fde069a-a7c7-467e-bc1f-ee5820163206"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:47:52.970857 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:52.970824 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fde069a-a7c7-467e-bc1f-ee5820163206-istio-podinfo\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:47:52.970857 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:52.970851 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fde069a-a7c7-467e-bc1f-ee5820163206-istio-envoy\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:47:52.970857 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:52.970862 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5tnw\" (UniqueName: \"kubernetes.io/projected/0fde069a-a7c7-467e-bc1f-ee5820163206-kube-api-access-r5tnw\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:47:52.971091 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:52.970873 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fde069a-a7c7-467e-bc1f-ee5820163206-istio-token\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:47:52.971091 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:52.970881 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fde069a-a7c7-467e-bc1f-ee5820163206-workload-socket\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:47:53.486480 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:53.486446 2555 generic.go:356] "Generic (PLEG): container finished" podID="0fde069a-a7c7-467e-bc1f-ee5820163206" containerID="420e69178ebde9fbd8e254be192126e2874d25f87f85265327f9c93a80907120" exitCode=0 Sep 08 18:47:53.486657 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:53.486500 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" Sep 08 18:47:53.486657 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:53.486542 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" event={"ID":"0fde069a-a7c7-467e-bc1f-ee5820163206","Type":"ContainerDied","Data":"420e69178ebde9fbd8e254be192126e2874d25f87f85265327f9c93a80907120"} Sep 08 18:47:53.486657 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:53.486601 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb" event={"ID":"0fde069a-a7c7-467e-bc1f-ee5820163206","Type":"ContainerDied","Data":"fd65e9afa49dfe22cfc617e4c18ebca576fa9fd3e082057bf4eb6e75eea5d5d4"} Sep 08 18:47:53.486657 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:53.486630 2555 scope.go:117] "RemoveContainer" containerID="420e69178ebde9fbd8e254be192126e2874d25f87f85265327f9c93a80907120" Sep 08 18:47:53.499776 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:53.499758 2555 scope.go:117] "RemoveContainer" containerID="85ae468f63338512e70158f043b275aa4656084103b76a33b0a12f51fad5ad83" Sep 08 18:47:53.508982 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:53.508965 2555 scope.go:117] "RemoveContainer" containerID="420e69178ebde9fbd8e254be192126e2874d25f87f85265327f9c93a80907120" Sep 08 18:47:53.509241 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:47:53.509222 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"420e69178ebde9fbd8e254be192126e2874d25f87f85265327f9c93a80907120\": container with ID starting with 420e69178ebde9fbd8e254be192126e2874d25f87f85265327f9c93a80907120 not found: ID does not exist" containerID="420e69178ebde9fbd8e254be192126e2874d25f87f85265327f9c93a80907120" Sep 08 18:47:53.509305 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:53.509249 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"420e69178ebde9fbd8e254be192126e2874d25f87f85265327f9c93a80907120"} err="failed to get container status \"420e69178ebde9fbd8e254be192126e2874d25f87f85265327f9c93a80907120\": rpc error: code = NotFound desc = could not find container \"420e69178ebde9fbd8e254be192126e2874d25f87f85265327f9c93a80907120\": container with ID starting with 420e69178ebde9fbd8e254be192126e2874d25f87f85265327f9c93a80907120 not found: ID does not exist" Sep 08 18:47:53.509305 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:53.509264 2555 scope.go:117] "RemoveContainer" containerID="85ae468f63338512e70158f043b275aa4656084103b76a33b0a12f51fad5ad83" Sep 08 18:47:53.509516 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:47:53.509502 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85ae468f63338512e70158f043b275aa4656084103b76a33b0a12f51fad5ad83\": container with ID starting with 85ae468f63338512e70158f043b275aa4656084103b76a33b0a12f51fad5ad83 not found: ID does not exist" containerID="85ae468f63338512e70158f043b275aa4656084103b76a33b0a12f51fad5ad83" Sep 08 18:47:53.509560 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:53.509523 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85ae468f63338512e70158f043b275aa4656084103b76a33b0a12f51fad5ad83"} err="failed to get container status \"85ae468f63338512e70158f043b275aa4656084103b76a33b0a12f51fad5ad83\": rpc error: code = NotFound desc = could not find container \"85ae468f63338512e70158f043b275aa4656084103b76a33b0a12f51fad5ad83\": container with ID starting with 85ae468f63338512e70158f043b275aa4656084103b76a33b0a12f51fad5ad83 not found: ID does not exist" Sep 08 18:47:53.509937 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:53.509915 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb"] Sep 08 18:47:53.513499 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:53.513480 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-ff349c-openshift-default-7bff5c6cbb-lbztb"] Sep 08 18:47:54.325774 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:54.325739 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0fde069a-a7c7-467e-bc1f-ee5820163206" path="/var/lib/kubelet/pods/0fde069a-a7c7-467e-bc1f-ee5820163206/volumes" Sep 08 18:47:58.069022 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:47:58.068971 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:48:03.621573 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:03.621536 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:48:03.649029 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:03.648993 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-179a6f-575f9bc89c-ggs9b"] Sep 08 18:48:03.649278 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:03.649248 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-179a6f-575f9bc89c-ggs9b" podUID="d507f7eb-bb08-4147-924d-f02563c7d445" containerName="payload-processing" containerID="cri-o://72f1cf688ab10223d9abc72fec9cd820aebf743b2989964328a97d8c2fa1acde" gracePeriod=30 Sep 08 18:48:03.654660 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:03.654632 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-179a6f-68c5f896cd-wg7q2"] Sep 08 18:48:03.654949 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:03.654901 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-179a6f-68c5f896cd-wg7q2" podUID="788a9706-0aa4-4918-8c84-43e372f7db05" containerName="payload-pre-processing" containerID="cri-o://161c3361a4cf1f8b5d2f1f8a84573ef28a1feac110b732a450825954553aa796" gracePeriod=30 Sep 08 18:48:03.689609 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:03.689582 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:48:03.755707 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:03.755650 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:48:03.856225 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:03.856190 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:48:03.936035 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:03.936007 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-179a6f-575f9bc89c-ggs9b" Sep 08 18:48:03.938816 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:03.938791 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-179a6f-68c5f896cd-wg7q2" Sep 08 18:48:04.082741 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:04.082663 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d507f7eb-bb08-4147-924d-f02563c7d445-plugins-config-volume\") pod \"d507f7eb-bb08-4147-924d-f02563c7d445\" (UID: \"d507f7eb-bb08-4147-924d-f02563c7d445\") " Sep 08 18:48:04.082741 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:04.082742 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zxbhf\" (UniqueName: \"kubernetes.io/projected/788a9706-0aa4-4918-8c84-43e372f7db05-kube-api-access-zxbhf\") pod \"788a9706-0aa4-4918-8c84-43e372f7db05\" (UID: \"788a9706-0aa4-4918-8c84-43e372f7db05\") " Sep 08 18:48:04.083002 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:04.082768 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hbtq4\" (UniqueName: \"kubernetes.io/projected/d507f7eb-bb08-4147-924d-f02563c7d445-kube-api-access-hbtq4\") pod \"d507f7eb-bb08-4147-924d-f02563c7d445\" (UID: \"d507f7eb-bb08-4147-924d-f02563c7d445\") " Sep 08 18:48:04.083002 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:04.082872 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/788a9706-0aa4-4918-8c84-43e372f7db05-plugins-config-volume\") pod \"788a9706-0aa4-4918-8c84-43e372f7db05\" (UID: \"788a9706-0aa4-4918-8c84-43e372f7db05\") " Sep 08 18:48:04.083241 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:04.083207 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d507f7eb-bb08-4147-924d-f02563c7d445-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d507f7eb-bb08-4147-924d-f02563c7d445" (UID: "d507f7eb-bb08-4147-924d-f02563c7d445"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:48:04.083338 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:04.083264 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/788a9706-0aa4-4918-8c84-43e372f7db05-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "788a9706-0aa4-4918-8c84-43e372f7db05" (UID: "788a9706-0aa4-4918-8c84-43e372f7db05"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:48:04.085086 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:04.085061 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/788a9706-0aa4-4918-8c84-43e372f7db05-kube-api-access-zxbhf" (OuterVolumeSpecName: "kube-api-access-zxbhf") pod "788a9706-0aa4-4918-8c84-43e372f7db05" (UID: "788a9706-0aa4-4918-8c84-43e372f7db05"). InnerVolumeSpecName "kube-api-access-zxbhf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:48:04.085205 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:04.085191 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d507f7eb-bb08-4147-924d-f02563c7d445-kube-api-access-hbtq4" (OuterVolumeSpecName: "kube-api-access-hbtq4") pod "d507f7eb-bb08-4147-924d-f02563c7d445" (UID: "d507f7eb-bb08-4147-924d-f02563c7d445"). InnerVolumeSpecName "kube-api-access-hbtq4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:48:04.168237 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:04.168133 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:48:04.184437 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:04.184400 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/788a9706-0aa4-4918-8c84-43e372f7db05-plugins-config-volume\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:48:04.184437 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:04.184434 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d507f7eb-bb08-4147-924d-f02563c7d445-plugins-config-volume\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:48:04.184437 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:04.184443 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxbhf\" (UniqueName: \"kubernetes.io/projected/788a9706-0aa4-4918-8c84-43e372f7db05-kube-api-access-zxbhf\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:48:04.184619 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:04.184451 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hbtq4\" (UniqueName: \"kubernetes.io/projected/d507f7eb-bb08-4147-924d-f02563c7d445-kube-api-access-hbtq4\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:48:04.563798 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:04.563763 2555 generic.go:356] "Generic (PLEG): container finished" podID="d507f7eb-bb08-4147-924d-f02563c7d445" containerID="72f1cf688ab10223d9abc72fec9cd820aebf743b2989964328a97d8c2fa1acde" exitCode=0 Sep 08 18:48:04.563982 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:04.563816 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-179a6f-575f9bc89c-ggs9b" Sep 08 18:48:04.563982 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:04.563852 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-179a6f-575f9bc89c-ggs9b" event={"ID":"d507f7eb-bb08-4147-924d-f02563c7d445","Type":"ContainerDied","Data":"72f1cf688ab10223d9abc72fec9cd820aebf743b2989964328a97d8c2fa1acde"} Sep 08 18:48:04.563982 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:04.563907 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-179a6f-575f9bc89c-ggs9b" event={"ID":"d507f7eb-bb08-4147-924d-f02563c7d445","Type":"ContainerDied","Data":"139b647f902c01a3bbafac39325b4b83ce920ebe49290d992ca0375463a87793"} Sep 08 18:48:04.563982 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:04.563931 2555 scope.go:117] "RemoveContainer" containerID="72f1cf688ab10223d9abc72fec9cd820aebf743b2989964328a97d8c2fa1acde" Sep 08 18:48:04.565356 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:04.565331 2555 generic.go:356] "Generic (PLEG): container finished" podID="788a9706-0aa4-4918-8c84-43e372f7db05" containerID="161c3361a4cf1f8b5d2f1f8a84573ef28a1feac110b732a450825954553aa796" exitCode=0 Sep 08 18:48:04.565442 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:04.565374 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-179a6f-68c5f896cd-wg7q2" Sep 08 18:48:04.565442 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:04.565399 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-179a6f-68c5f896cd-wg7q2" event={"ID":"788a9706-0aa4-4918-8c84-43e372f7db05","Type":"ContainerDied","Data":"161c3361a4cf1f8b5d2f1f8a84573ef28a1feac110b732a450825954553aa796"} Sep 08 18:48:04.565532 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:04.565455 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-179a6f-68c5f896cd-wg7q2" event={"ID":"788a9706-0aa4-4918-8c84-43e372f7db05","Type":"ContainerDied","Data":"52ab5657d7703267be0e151fd7c11ddf42ed38e6d9463d8ec6f3c394c09dce73"} Sep 08 18:48:04.574566 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:04.574545 2555 scope.go:117] "RemoveContainer" containerID="72f1cf688ab10223d9abc72fec9cd820aebf743b2989964328a97d8c2fa1acde" Sep 08 18:48:04.574925 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:48:04.574905 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72f1cf688ab10223d9abc72fec9cd820aebf743b2989964328a97d8c2fa1acde\": container with ID starting with 72f1cf688ab10223d9abc72fec9cd820aebf743b2989964328a97d8c2fa1acde not found: ID does not exist" containerID="72f1cf688ab10223d9abc72fec9cd820aebf743b2989964328a97d8c2fa1acde" Sep 08 18:48:04.574979 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:04.574936 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72f1cf688ab10223d9abc72fec9cd820aebf743b2989964328a97d8c2fa1acde"} err="failed to get container status \"72f1cf688ab10223d9abc72fec9cd820aebf743b2989964328a97d8c2fa1acde\": rpc error: code = NotFound desc = could not find container \"72f1cf688ab10223d9abc72fec9cd820aebf743b2989964328a97d8c2fa1acde\": container with ID starting with 72f1cf688ab10223d9abc72fec9cd820aebf743b2989964328a97d8c2fa1acde not found: ID does not exist" Sep 08 18:48:04.574979 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:04.574955 2555 scope.go:117] "RemoveContainer" containerID="161c3361a4cf1f8b5d2f1f8a84573ef28a1feac110b732a450825954553aa796" Sep 08 18:48:04.585651 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:04.585558 2555 scope.go:117] "RemoveContainer" containerID="161c3361a4cf1f8b5d2f1f8a84573ef28a1feac110b732a450825954553aa796" Sep 08 18:48:04.586105 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:48:04.586073 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"161c3361a4cf1f8b5d2f1f8a84573ef28a1feac110b732a450825954553aa796\": container with ID starting with 161c3361a4cf1f8b5d2f1f8a84573ef28a1feac110b732a450825954553aa796 not found: ID does not exist" containerID="161c3361a4cf1f8b5d2f1f8a84573ef28a1feac110b732a450825954553aa796" Sep 08 18:48:04.586203 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:04.586116 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"161c3361a4cf1f8b5d2f1f8a84573ef28a1feac110b732a450825954553aa796"} err="failed to get container status \"161c3361a4cf1f8b5d2f1f8a84573ef28a1feac110b732a450825954553aa796\": rpc error: code = NotFound desc = could not find container \"161c3361a4cf1f8b5d2f1f8a84573ef28a1feac110b732a450825954553aa796\": container with ID starting with 161c3361a4cf1f8b5d2f1f8a84573ef28a1feac110b732a450825954553aa796 not found: ID does not exist" Sep 08 18:48:04.586401 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:04.586383 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-179a6f-575f9bc89c-ggs9b"] Sep 08 18:48:04.589157 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:04.589132 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-179a6f-575f9bc89c-ggs9b"] Sep 08 18:48:04.597678 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:04.597647 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-179a6f-68c5f896cd-wg7q2"] Sep 08 18:48:04.601582 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:04.601544 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-179a6f-68c5f896cd-wg7q2"] Sep 08 18:48:06.324554 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:06.324519 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="788a9706-0aa4-4918-8c84-43e372f7db05" path="/var/lib/kubelet/pods/788a9706-0aa4-4918-8c84-43e372f7db05/volumes" Sep 08 18:48:06.324960 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:06.324936 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d507f7eb-bb08-4147-924d-f02563c7d445" path="/var/lib/kubelet/pods/d507f7eb-bb08-4147-924d-f02563c7d445/volumes" Sep 08 18:48:08.023363 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:08.023326 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:48:09.427043 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:09.427015 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:48:10.081608 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:10.081575 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:48:10.145950 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:10.145920 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:48:10.330625 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:10.330595 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:48:10.852238 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:10.852203 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:48:21.014848 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.014815 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:48:21.133130 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.133089 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:48:21.134199 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.134176 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-3b04d6-29814885-vqkhv"] Sep 08 18:48:21.136599 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.136572 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-3b04d6-29814885-vqkhv"] Sep 08 18:48:21.155828 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.155798 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-3b04d6-5f657ddf9f-6vk9h"] Sep 08 18:48:21.156080 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.156046 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3b04d6-5f657ddf9f-6vk9h" podUID="7492083a-66f1-4624-9662-86ba98931e80" containerName="payload-processing" containerID="cri-o://98e367d8f6e1fcbde685bdee8c10eb8c7745df84974cf44284586bf68f8b353a" gracePeriod=30 Sep 08 18:48:21.166938 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.166912 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b04d6-86d98c5cf8-jx6hr"] Sep 08 18:48:21.167166 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.167140 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b04d6-86d98c5cf8-jx6hr" podUID="a5ee600c-bcc2-476c-beb8-cf617f8008fb" containerName="payload-pre-processing" containerID="cri-o://fea38bc5d1ce3f0ee2a365a0ff2d5dfef567076ea65170bb35843536242a9373" gracePeriod=30 Sep 08 18:48:21.196585 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.196560 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:48:21.252333 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.252291 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:48:21.321067 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.321026 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:48:21.382945 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.382911 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:48:21.427483 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.427463 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3b04d6-5f657ddf9f-6vk9h" Sep 08 18:48:21.439383 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.439360 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4vgqs\" (UniqueName: \"kubernetes.io/projected/7492083a-66f1-4624-9662-86ba98931e80-kube-api-access-4vgqs\") pod \"7492083a-66f1-4624-9662-86ba98931e80\" (UID: \"7492083a-66f1-4624-9662-86ba98931e80\") " Sep 08 18:48:21.439507 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.439404 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7492083a-66f1-4624-9662-86ba98931e80-plugins-config-volume\") pod \"7492083a-66f1-4624-9662-86ba98931e80\" (UID: \"7492083a-66f1-4624-9662-86ba98931e80\") " Sep 08 18:48:21.439794 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.439772 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7492083a-66f1-4624-9662-86ba98931e80-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7492083a-66f1-4624-9662-86ba98931e80" (UID: "7492083a-66f1-4624-9662-86ba98931e80"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:48:21.441640 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.441599 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7492083a-66f1-4624-9662-86ba98931e80-kube-api-access-4vgqs" (OuterVolumeSpecName: "kube-api-access-4vgqs") pod "7492083a-66f1-4624-9662-86ba98931e80" (UID: "7492083a-66f1-4624-9662-86ba98931e80"). InnerVolumeSpecName "kube-api-access-4vgqs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:48:21.441640 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.441626 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b04d6-86d98c5cf8-jx6hr" Sep 08 18:48:21.540009 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.539984 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5ee600c-bcc2-476c-beb8-cf617f8008fb-plugins-config-volume\") pod \"a5ee600c-bcc2-476c-beb8-cf617f8008fb\" (UID: \"a5ee600c-bcc2-476c-beb8-cf617f8008fb\") " Sep 08 18:48:21.540157 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.540076 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b8sv6\" (UniqueName: \"kubernetes.io/projected/a5ee600c-bcc2-476c-beb8-cf617f8008fb-kube-api-access-b8sv6\") pod \"a5ee600c-bcc2-476c-beb8-cf617f8008fb\" (UID: \"a5ee600c-bcc2-476c-beb8-cf617f8008fb\") " Sep 08 18:48:21.540244 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.540224 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4vgqs\" (UniqueName: \"kubernetes.io/projected/7492083a-66f1-4624-9662-86ba98931e80-kube-api-access-4vgqs\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:48:21.540244 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.540236 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7492083a-66f1-4624-9662-86ba98931e80-plugins-config-volume\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:48:21.540406 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.540381 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5ee600c-bcc2-476c-beb8-cf617f8008fb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a5ee600c-bcc2-476c-beb8-cf617f8008fb" (UID: "a5ee600c-bcc2-476c-beb8-cf617f8008fb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:48:21.542283 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.542260 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5ee600c-bcc2-476c-beb8-cf617f8008fb-kube-api-access-b8sv6" (OuterVolumeSpecName: "kube-api-access-b8sv6") pod "a5ee600c-bcc2-476c-beb8-cf617f8008fb" (UID: "a5ee600c-bcc2-476c-beb8-cf617f8008fb"). InnerVolumeSpecName "kube-api-access-b8sv6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:48:21.641034 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.641010 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5ee600c-bcc2-476c-beb8-cf617f8008fb-plugins-config-volume\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:48:21.641034 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.641030 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8sv6\" (UniqueName: \"kubernetes.io/projected/a5ee600c-bcc2-476c-beb8-cf617f8008fb-kube-api-access-b8sv6\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:48:21.671049 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.671021 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:48:21.681954 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.681922 2555 generic.go:356] "Generic (PLEG): container finished" podID="a5ee600c-bcc2-476c-beb8-cf617f8008fb" containerID="fea38bc5d1ce3f0ee2a365a0ff2d5dfef567076ea65170bb35843536242a9373" exitCode=0 Sep 08 18:48:21.682067 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.681968 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b04d6-86d98c5cf8-jx6hr" Sep 08 18:48:21.682067 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.681979 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b04d6-86d98c5cf8-jx6hr" event={"ID":"a5ee600c-bcc2-476c-beb8-cf617f8008fb","Type":"ContainerDied","Data":"fea38bc5d1ce3f0ee2a365a0ff2d5dfef567076ea65170bb35843536242a9373"} Sep 08 18:48:21.682067 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.682039 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b04d6-86d98c5cf8-jx6hr" event={"ID":"a5ee600c-bcc2-476c-beb8-cf617f8008fb","Type":"ContainerDied","Data":"880388f148ee2be07d7c7a3322825103bac080dace76e844299f996e7d7db65f"} Sep 08 18:48:21.682227 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.682069 2555 scope.go:117] "RemoveContainer" containerID="fea38bc5d1ce3f0ee2a365a0ff2d5dfef567076ea65170bb35843536242a9373" Sep 08 18:48:21.683536 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.683486 2555 generic.go:356] "Generic (PLEG): container finished" podID="7492083a-66f1-4624-9662-86ba98931e80" containerID="98e367d8f6e1fcbde685bdee8c10eb8c7745df84974cf44284586bf68f8b353a" exitCode=0 Sep 08 18:48:21.683536 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.683547 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3b04d6-5f657ddf9f-6vk9h" event={"ID":"7492083a-66f1-4624-9662-86ba98931e80","Type":"ContainerDied","Data":"98e367d8f6e1fcbde685bdee8c10eb8c7745df84974cf44284586bf68f8b353a"} Sep 08 18:48:21.683536 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.683580 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3b04d6-5f657ddf9f-6vk9h" event={"ID":"7492083a-66f1-4624-9662-86ba98931e80","Type":"ContainerDied","Data":"4a5e9761fee57754a39a2c297db591165eb70cf1d79ec93adff52e08e9ab3a36"} Sep 08 18:48:21.686963 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.683953 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3b04d6-5f657ddf9f-6vk9h" Sep 08 18:48:21.699564 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.699545 2555 scope.go:117] "RemoveContainer" containerID="fea38bc5d1ce3f0ee2a365a0ff2d5dfef567076ea65170bb35843536242a9373" Sep 08 18:48:21.699885 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:48:21.699866 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fea38bc5d1ce3f0ee2a365a0ff2d5dfef567076ea65170bb35843536242a9373\": container with ID starting with fea38bc5d1ce3f0ee2a365a0ff2d5dfef567076ea65170bb35843536242a9373 not found: ID does not exist" containerID="fea38bc5d1ce3f0ee2a365a0ff2d5dfef567076ea65170bb35843536242a9373" Sep 08 18:48:21.699954 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.699893 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fea38bc5d1ce3f0ee2a365a0ff2d5dfef567076ea65170bb35843536242a9373"} err="failed to get container status \"fea38bc5d1ce3f0ee2a365a0ff2d5dfef567076ea65170bb35843536242a9373\": rpc error: code = NotFound desc = could not find container \"fea38bc5d1ce3f0ee2a365a0ff2d5dfef567076ea65170bb35843536242a9373\": container with ID starting with fea38bc5d1ce3f0ee2a365a0ff2d5dfef567076ea65170bb35843536242a9373 not found: ID does not exist" Sep 08 18:48:21.699954 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.699909 2555 scope.go:117] "RemoveContainer" containerID="98e367d8f6e1fcbde685bdee8c10eb8c7745df84974cf44284586bf68f8b353a" Sep 08 18:48:21.707917 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.707890 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b04d6-86d98c5cf8-jx6hr"] Sep 08 18:48:21.709289 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.709270 2555 scope.go:117] "RemoveContainer" containerID="98e367d8f6e1fcbde685bdee8c10eb8c7745df84974cf44284586bf68f8b353a" Sep 08 18:48:21.709575 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:48:21.709544 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98e367d8f6e1fcbde685bdee8c10eb8c7745df84974cf44284586bf68f8b353a\": container with ID starting with 98e367d8f6e1fcbde685bdee8c10eb8c7745df84974cf44284586bf68f8b353a not found: ID does not exist" containerID="98e367d8f6e1fcbde685bdee8c10eb8c7745df84974cf44284586bf68f8b353a" Sep 08 18:48:21.709668 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.709586 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98e367d8f6e1fcbde685bdee8c10eb8c7745df84974cf44284586bf68f8b353a"} err="failed to get container status \"98e367d8f6e1fcbde685bdee8c10eb8c7745df84974cf44284586bf68f8b353a\": rpc error: code = NotFound desc = could not find container \"98e367d8f6e1fcbde685bdee8c10eb8c7745df84974cf44284586bf68f8b353a\": container with ID starting with 98e367d8f6e1fcbde685bdee8c10eb8c7745df84974cf44284586bf68f8b353a not found: ID does not exist" Sep 08 18:48:21.710319 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.710302 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b04d6-86d98c5cf8-jx6hr"] Sep 08 18:48:21.720557 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.720536 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-3b04d6-5f657ddf9f-6vk9h"] Sep 08 18:48:21.722910 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:21.722893 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-3b04d6-5f657ddf9f-6vk9h"] Sep 08 18:48:22.324758 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:22.324727 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7492083a-66f1-4624-9662-86ba98931e80" path="/var/lib/kubelet/pods/7492083a-66f1-4624-9662-86ba98931e80/volumes" Sep 08 18:48:22.325145 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:22.325109 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5ee600c-bcc2-476c-beb8-cf617f8008fb" path="/var/lib/kubelet/pods/a5ee600c-bcc2-476c-beb8-cf617f8008fb/volumes" Sep 08 18:48:22.325349 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:22.325336 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da3000a7-1ea1-4009-97e9-62483f7d70db" path="/var/lib/kubelet/pods/da3000a7-1ea1-4009-97e9-62483f7d70db/volumes" Sep 08 18:48:22.418944 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:22.418906 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3b04d6-5f657ddf9f-6vk9h" podUID="7492083a-66f1-4624-9662-86ba98931e80" containerName="payload-processing" probeResult="failure" output="dial tcp 10.132.0.63:9004: i/o timeout" Sep 08 18:48:22.419119 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:22.418910 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3b04d6-86d98c5cf8-jx6hr" podUID="a5ee600c-bcc2-476c-beb8-cf617f8008fb" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.64:9004: i/o timeout" Sep 08 18:48:25.323120 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:25.323083 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:48:27.236102 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:27.236066 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:48:27.296475 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:27.296444 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:48:27.480413 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:27.480378 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:48:27.818029 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:27.817989 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-951c3e-nph2w"] Sep 08 18:48:27.818404 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:27.818391 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0fde069a-a7c7-467e-bc1f-ee5820163206" containerName="istio-proxy" Sep 08 18:48:27.818451 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:27.818405 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fde069a-a7c7-467e-bc1f-ee5820163206" containerName="istio-proxy" Sep 08 18:48:27.818451 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:27.818418 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7492083a-66f1-4624-9662-86ba98931e80" containerName="payload-processing" Sep 08 18:48:27.818451 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:27.818424 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="7492083a-66f1-4624-9662-86ba98931e80" containerName="payload-processing" Sep 08 18:48:27.818451 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:27.818436 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0fde069a-a7c7-467e-bc1f-ee5820163206" containerName="istio-proxy" Sep 08 18:48:27.818451 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:27.818441 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fde069a-a7c7-467e-bc1f-ee5820163206" containerName="istio-proxy" Sep 08 18:48:27.818607 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:27.818458 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d507f7eb-bb08-4147-924d-f02563c7d445" containerName="payload-processing" Sep 08 18:48:27.818607 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:27.818465 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="d507f7eb-bb08-4147-924d-f02563c7d445" containerName="payload-processing" Sep 08 18:48:27.818607 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:27.818478 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5ee600c-bcc2-476c-beb8-cf617f8008fb" containerName="payload-pre-processing" Sep 08 18:48:27.818607 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:27.818483 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5ee600c-bcc2-476c-beb8-cf617f8008fb" containerName="payload-pre-processing" Sep 08 18:48:27.818607 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:27.818490 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c56b5fa2-71aa-4315-bc0a-6d4f24c9067f" containerName="payload-processing" Sep 08 18:48:27.818607 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:27.818495 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="c56b5fa2-71aa-4315-bc0a-6d4f24c9067f" containerName="payload-processing" Sep 08 18:48:27.818607 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:27.818502 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="788a9706-0aa4-4918-8c84-43e372f7db05" containerName="payload-pre-processing" Sep 08 18:48:27.818607 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:27.818507 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="788a9706-0aa4-4918-8c84-43e372f7db05" containerName="payload-pre-processing" Sep 08 18:48:27.818607 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:27.818567 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="7492083a-66f1-4624-9662-86ba98931e80" containerName="payload-processing" Sep 08 18:48:27.818607 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:27.818576 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="0fde069a-a7c7-467e-bc1f-ee5820163206" containerName="istio-proxy" Sep 08 18:48:27.818607 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:27.818582 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="0fde069a-a7c7-467e-bc1f-ee5820163206" containerName="istio-proxy" Sep 08 18:48:27.818607 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:27.818588 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="788a9706-0aa4-4918-8c84-43e372f7db05" containerName="payload-pre-processing" Sep 08 18:48:27.818607 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:27.818598 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="d507f7eb-bb08-4147-924d-f02563c7d445" containerName="payload-processing" Sep 08 18:48:27.818607 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:27.818604 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="c56b5fa2-71aa-4315-bc0a-6d4f24c9067f" containerName="payload-processing" Sep 08 18:48:27.819069 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:27.818614 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5ee600c-bcc2-476c-beb8-cf617f8008fb" containerName="payload-pre-processing" Sep 08 18:48:27.823247 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:27.823229 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-951c3e-nph2w" Sep 08 18:48:27.825433 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:27.825407 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 08 18:48:27.825580 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:27.825413 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g6q8h\"" Sep 08 18:48:27.828968 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:27.828942 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-951c3e-nph2w"] Sep 08 18:48:27.892323 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:27.892285 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8271c400-ce4c-46e9-b7a8-f6eea03901c5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-951c3e-nph2w\" (UID: \"8271c400-ce4c-46e9-b7a8-f6eea03901c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-951c3e-nph2w" Sep 08 18:48:27.923338 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:27.923308 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:48:27.993771 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:27.993729 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8271c400-ce4c-46e9-b7a8-f6eea03901c5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-951c3e-nph2w\" (UID: \"8271c400-ce4c-46e9-b7a8-f6eea03901c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-951c3e-nph2w" Sep 08 18:48:27.994392 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:27.994366 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8271c400-ce4c-46e9-b7a8-f6eea03901c5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-951c3e-nph2w\" (UID: \"8271c400-ce4c-46e9-b7a8-f6eea03901c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-951c3e-nph2w" Sep 08 18:48:28.136333 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:28.136240 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-951c3e-nph2w" Sep 08 18:48:28.415984 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:28.415955 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-951c3e-nph2w"] Sep 08 18:48:28.417851 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:48:28.417826 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8271c400_ce4c_46e9_b7a8_f6eea03901c5.slice/crio-a91ff96a404669d7072161bed9bd2b3b52eea52725dc085e3f94291a815f7664 WatchSource:0}: Error finding container a91ff96a404669d7072161bed9bd2b3b52eea52725dc085e3f94291a815f7664: Status 404 returned error can't find the container with id a91ff96a404669d7072161bed9bd2b3b52eea52725dc085e3f94291a815f7664 Sep 08 18:48:28.729513 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:28.729470 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-951c3e-nph2w" event={"ID":"8271c400-ce4c-46e9-b7a8-f6eea03901c5","Type":"ContainerStarted","Data":"ac6d5f3ed94fb3fee2a57ee7003cad1c75dfc54ce53959608ad6244527bb6ae0"} Sep 08 18:48:28.729745 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:28.729530 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-951c3e-nph2w" event={"ID":"8271c400-ce4c-46e9-b7a8-f6eea03901c5","Type":"ContainerStarted","Data":"a91ff96a404669d7072161bed9bd2b3b52eea52725dc085e3f94291a815f7664"} Sep 08 18:48:28.743888 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:28.743837 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-951c3e-nph2w" podStartSLOduration=1.743819123 podStartE2EDuration="1.743819123s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:48:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:48:28.741770399 +0000 UTC m=+1490.947755800" watchObservedRunningTime="2026-09-08 18:48:28.743819123 +0000 UTC m=+1490.949804520" Sep 08 18:48:29.737446 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:29.737412 2555 generic.go:356] "Generic (PLEG): container finished" podID="8271c400-ce4c-46e9-b7a8-f6eea03901c5" containerID="ac6d5f3ed94fb3fee2a57ee7003cad1c75dfc54ce53959608ad6244527bb6ae0" exitCode=0 Sep 08 18:48:29.737890 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:29.737462 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-951c3e-nph2w" event={"ID":"8271c400-ce4c-46e9-b7a8-f6eea03901c5","Type":"ContainerDied","Data":"ac6d5f3ed94fb3fee2a57ee7003cad1c75dfc54ce53959608ad6244527bb6ae0"} Sep 08 18:48:30.879599 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:30.879570 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-951c3e-nph2w" Sep 08 18:48:30.922258 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:30.922225 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8271c400-ce4c-46e9-b7a8-f6eea03901c5-maas-api-service-ca\") pod \"8271c400-ce4c-46e9-b7a8-f6eea03901c5\" (UID: \"8271c400-ce4c-46e9-b7a8-f6eea03901c5\") " Sep 08 18:48:30.922595 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:30.922570 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8271c400-ce4c-46e9-b7a8-f6eea03901c5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8271c400-ce4c-46e9-b7a8-f6eea03901c5" (UID: "8271c400-ce4c-46e9-b7a8-f6eea03901c5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:48:31.023766 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:31.023650 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8271c400-ce4c-46e9-b7a8-f6eea03901c5-maas-api-service-ca\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:48:31.750539 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:31.750513 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-951c3e-nph2w" Sep 08 18:48:31.750761 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:31.750551 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-951c3e-nph2w" event={"ID":"8271c400-ce4c-46e9-b7a8-f6eea03901c5","Type":"ContainerDied","Data":"a91ff96a404669d7072161bed9bd2b3b52eea52725dc085e3f94291a815f7664"} Sep 08 18:48:31.750761 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:31.750595 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a91ff96a404669d7072161bed9bd2b3b52eea52725dc085e3f94291a815f7664" Sep 08 18:48:38.256762 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.256727 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-951c3e-6869d8c4dc-tl7px"] Sep 08 18:48:38.257340 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.256983 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-951c3e-6869d8c4dc-tl7px" podUID="80c36c09-e533-446b-82fe-28d44a96e1d8" containerName="payload-processing" containerID="cri-o://4ca09cf1ed95c220ac377c75d1187ca580f473ff042ca712e6d42ebfe3a11114" gracePeriod=30 Sep 08 18:48:38.260665 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.260634 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-951c3e-6657c964d6-l8bq9"] Sep 08 18:48:38.260922 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.260875 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-951c3e-6657c964d6-l8bq9" podUID="131d83ac-f065-488a-872b-c40c09cfccd5" containerName="payload-pre-processing" containerID="cri-o://39c723be7c296a037f5e41bee8dc1e6b9e82532d0ff6f3bb562f379e2904fc5b" gracePeriod=30 Sep 08 18:48:38.393673 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.393616 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-951c3e-nph2w"] Sep 08 18:48:38.398363 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.398329 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-951c3e-nph2w"] Sep 08 18:48:38.433088 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.432489 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9n28f_911c61b8-766b-4ae8-b046-784e22e85889/console-operator/1.log" Sep 08 18:48:38.439888 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.439866 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9n28f_911c61b8-766b-4ae8-b046-784e22e85889/console-operator/1.log" Sep 08 18:48:38.535509 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.535478 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-951c3e-6869d8c4dc-tl7px" Sep 08 18:48:38.538530 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.538514 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-951c3e-6657c964d6-l8bq9" Sep 08 18:48:38.589760 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.589730 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/131d83ac-f065-488a-872b-c40c09cfccd5-plugins-config-volume\") pod \"131d83ac-f065-488a-872b-c40c09cfccd5\" (UID: \"131d83ac-f065-488a-872b-c40c09cfccd5\") " Sep 08 18:48:38.589925 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.589846 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xjgdg\" (UniqueName: \"kubernetes.io/projected/131d83ac-f065-488a-872b-c40c09cfccd5-kube-api-access-xjgdg\") pod \"131d83ac-f065-488a-872b-c40c09cfccd5\" (UID: \"131d83ac-f065-488a-872b-c40c09cfccd5\") " Sep 08 18:48:38.589996 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.589935 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ps96\" (UniqueName: \"kubernetes.io/projected/80c36c09-e533-446b-82fe-28d44a96e1d8-kube-api-access-8ps96\") pod \"80c36c09-e533-446b-82fe-28d44a96e1d8\" (UID: \"80c36c09-e533-446b-82fe-28d44a96e1d8\") " Sep 08 18:48:38.589996 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.589974 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80c36c09-e533-446b-82fe-28d44a96e1d8-plugins-config-volume\") pod \"80c36c09-e533-446b-82fe-28d44a96e1d8\" (UID: \"80c36c09-e533-446b-82fe-28d44a96e1d8\") " Sep 08 18:48:38.590147 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.590125 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/131d83ac-f065-488a-872b-c40c09cfccd5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "131d83ac-f065-488a-872b-c40c09cfccd5" (UID: "131d83ac-f065-488a-872b-c40c09cfccd5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:48:38.590354 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.590331 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/131d83ac-f065-488a-872b-c40c09cfccd5-plugins-config-volume\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:48:38.590424 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.590370 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80c36c09-e533-446b-82fe-28d44a96e1d8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "80c36c09-e533-446b-82fe-28d44a96e1d8" (UID: "80c36c09-e533-446b-82fe-28d44a96e1d8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:48:38.592011 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.591986 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/131d83ac-f065-488a-872b-c40c09cfccd5-kube-api-access-xjgdg" (OuterVolumeSpecName: "kube-api-access-xjgdg") pod "131d83ac-f065-488a-872b-c40c09cfccd5" (UID: "131d83ac-f065-488a-872b-c40c09cfccd5"). InnerVolumeSpecName "kube-api-access-xjgdg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:48:38.592565 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.592548 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80c36c09-e533-446b-82fe-28d44a96e1d8-kube-api-access-8ps96" (OuterVolumeSpecName: "kube-api-access-8ps96") pod "80c36c09-e533-446b-82fe-28d44a96e1d8" (UID: "80c36c09-e533-446b-82fe-28d44a96e1d8"). InnerVolumeSpecName "kube-api-access-8ps96". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:48:38.691737 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.691700 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ps96\" (UniqueName: \"kubernetes.io/projected/80c36c09-e533-446b-82fe-28d44a96e1d8-kube-api-access-8ps96\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:48:38.691737 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.691736 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80c36c09-e533-446b-82fe-28d44a96e1d8-plugins-config-volume\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:48:38.691737 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.691745 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjgdg\" (UniqueName: \"kubernetes.io/projected/131d83ac-f065-488a-872b-c40c09cfccd5-kube-api-access-xjgdg\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:48:38.794260 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.794230 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:48:38.795670 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.795649 2555 generic.go:356] "Generic (PLEG): container finished" podID="80c36c09-e533-446b-82fe-28d44a96e1d8" containerID="4ca09cf1ed95c220ac377c75d1187ca580f473ff042ca712e6d42ebfe3a11114" exitCode=0 Sep 08 18:48:38.795793 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.795712 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-951c3e-6869d8c4dc-tl7px" Sep 08 18:48:38.795793 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.795733 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-951c3e-6869d8c4dc-tl7px" event={"ID":"80c36c09-e533-446b-82fe-28d44a96e1d8","Type":"ContainerDied","Data":"4ca09cf1ed95c220ac377c75d1187ca580f473ff042ca712e6d42ebfe3a11114"} Sep 08 18:48:38.795793 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.795782 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-951c3e-6869d8c4dc-tl7px" event={"ID":"80c36c09-e533-446b-82fe-28d44a96e1d8","Type":"ContainerDied","Data":"5a767960663f6d8a2149561b6c92a4ddf77b1b50894ef1a51a260b7b4e3a0c1e"} Sep 08 18:48:38.795956 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.795807 2555 scope.go:117] "RemoveContainer" containerID="4ca09cf1ed95c220ac377c75d1187ca580f473ff042ca712e6d42ebfe3a11114" Sep 08 18:48:38.797177 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.797156 2555 generic.go:356] "Generic (PLEG): container finished" podID="131d83ac-f065-488a-872b-c40c09cfccd5" containerID="39c723be7c296a037f5e41bee8dc1e6b9e82532d0ff6f3bb562f379e2904fc5b" exitCode=0 Sep 08 18:48:38.797270 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.797192 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-951c3e-6657c964d6-l8bq9" Sep 08 18:48:38.797270 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.797239 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-951c3e-6657c964d6-l8bq9" event={"ID":"131d83ac-f065-488a-872b-c40c09cfccd5","Type":"ContainerDied","Data":"39c723be7c296a037f5e41bee8dc1e6b9e82532d0ff6f3bb562f379e2904fc5b"} Sep 08 18:48:38.797403 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.797277 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-951c3e-6657c964d6-l8bq9" event={"ID":"131d83ac-f065-488a-872b-c40c09cfccd5","Type":"ContainerDied","Data":"b7c0695b56c0d9c60e73dead011af08b8640267728bafb5ae5f6c7515aab95b7"} Sep 08 18:48:38.805174 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.805154 2555 scope.go:117] "RemoveContainer" containerID="4ca09cf1ed95c220ac377c75d1187ca580f473ff042ca712e6d42ebfe3a11114" Sep 08 18:48:38.805446 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:48:38.805427 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ca09cf1ed95c220ac377c75d1187ca580f473ff042ca712e6d42ebfe3a11114\": container with ID starting with 4ca09cf1ed95c220ac377c75d1187ca580f473ff042ca712e6d42ebfe3a11114 not found: ID does not exist" containerID="4ca09cf1ed95c220ac377c75d1187ca580f473ff042ca712e6d42ebfe3a11114" Sep 08 18:48:38.805494 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.805455 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ca09cf1ed95c220ac377c75d1187ca580f473ff042ca712e6d42ebfe3a11114"} err="failed to get container status \"4ca09cf1ed95c220ac377c75d1187ca580f473ff042ca712e6d42ebfe3a11114\": rpc error: code = NotFound desc = could not find container \"4ca09cf1ed95c220ac377c75d1187ca580f473ff042ca712e6d42ebfe3a11114\": container with ID starting with 4ca09cf1ed95c220ac377c75d1187ca580f473ff042ca712e6d42ebfe3a11114 not found: ID does not exist" Sep 08 18:48:38.805494 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.805471 2555 scope.go:117] "RemoveContainer" containerID="39c723be7c296a037f5e41bee8dc1e6b9e82532d0ff6f3bb562f379e2904fc5b" Sep 08 18:48:38.813476 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.813457 2555 scope.go:117] "RemoveContainer" containerID="39c723be7c296a037f5e41bee8dc1e6b9e82532d0ff6f3bb562f379e2904fc5b" Sep 08 18:48:38.813743 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:48:38.813726 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39c723be7c296a037f5e41bee8dc1e6b9e82532d0ff6f3bb562f379e2904fc5b\": container with ID starting with 39c723be7c296a037f5e41bee8dc1e6b9e82532d0ff6f3bb562f379e2904fc5b not found: ID does not exist" containerID="39c723be7c296a037f5e41bee8dc1e6b9e82532d0ff6f3bb562f379e2904fc5b" Sep 08 18:48:38.813805 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.813750 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39c723be7c296a037f5e41bee8dc1e6b9e82532d0ff6f3bb562f379e2904fc5b"} err="failed to get container status \"39c723be7c296a037f5e41bee8dc1e6b9e82532d0ff6f3bb562f379e2904fc5b\": rpc error: code = NotFound desc = could not find container \"39c723be7c296a037f5e41bee8dc1e6b9e82532d0ff6f3bb562f379e2904fc5b\": container with ID starting with 39c723be7c296a037f5e41bee8dc1e6b9e82532d0ff6f3bb562f379e2904fc5b not found: ID does not exist" Sep 08 18:48:38.820022 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.820004 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-951c3e-6657c964d6-l8bq9"] Sep 08 18:48:38.823751 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.823730 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-951c3e-6657c964d6-l8bq9"] Sep 08 18:48:38.833154 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.833135 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-951c3e-6869d8c4dc-tl7px"] Sep 08 18:48:38.838777 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.838758 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-951c3e-6869d8c4dc-tl7px"] Sep 08 18:48:38.894914 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:38.894888 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:48:40.324130 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:40.324094 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="131d83ac-f065-488a-872b-c40c09cfccd5" path="/var/lib/kubelet/pods/131d83ac-f065-488a-872b-c40c09cfccd5/volumes" Sep 08 18:48:40.324584 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:40.324430 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80c36c09-e533-446b-82fe-28d44a96e1d8" path="/var/lib/kubelet/pods/80c36c09-e533-446b-82fe-28d44a96e1d8/volumes" Sep 08 18:48:40.324728 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:40.324714 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8271c400-ce4c-46e9-b7a8-f6eea03901c5" path="/var/lib/kubelet/pods/8271c400-ce4c-46e9-b7a8-f6eea03901c5/volumes" Sep 08 18:48:41.921655 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:41.921619 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:48:44.519128 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:44.519091 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:48:44.620338 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:44.620305 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:48:44.778646 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:44.778563 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:48:51.785457 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:51.785413 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:48:58.018526 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:48:58.018487 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:49:02.197668 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:02.197574 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:49:03.140070 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:03.140037 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:49:03.553765 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:03.553726 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:49:03.842130 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:03.842047 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8"] Sep 08 18:49:03.842617 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:03.842598 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="131d83ac-f065-488a-872b-c40c09cfccd5" containerName="payload-pre-processing" Sep 08 18:49:03.842713 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:03.842634 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="131d83ac-f065-488a-872b-c40c09cfccd5" containerName="payload-pre-processing" Sep 08 18:49:03.842713 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:03.842661 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8271c400-ce4c-46e9-b7a8-f6eea03901c5" containerName="revoke-keys" Sep 08 18:49:03.842713 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:03.842669 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="8271c400-ce4c-46e9-b7a8-f6eea03901c5" containerName="revoke-keys" Sep 08 18:49:03.842713 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:03.842708 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80c36c09-e533-446b-82fe-28d44a96e1d8" containerName="payload-processing" Sep 08 18:49:03.842876 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:03.842718 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="80c36c09-e533-446b-82fe-28d44a96e1d8" containerName="payload-processing" Sep 08 18:49:03.842876 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:03.842819 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="8271c400-ce4c-46e9-b7a8-f6eea03901c5" containerName="revoke-keys" Sep 08 18:49:03.842876 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:03.842834 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="131d83ac-f065-488a-872b-c40c09cfccd5" containerName="payload-pre-processing" Sep 08 18:49:03.842876 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:03.842844 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="80c36c09-e533-446b-82fe-28d44a96e1d8" containerName="payload-processing" Sep 08 18:49:03.847646 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:03.847625 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:49:03.850145 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:03.850125 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-bc23ef-openshift-default-dockercfg-s4h9b\"" Sep 08 18:49:03.858944 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:03.858920 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8"] Sep 08 18:49:03.880812 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:03.880778 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:49:04.018018 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:04.017988 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02bbe225-8f61-4fff-af3f-37c464881328-istio-token\") pod \"e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8\" (UID: \"02bbe225-8f61-4fff-af3f-37c464881328\") " pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:49:04.018197 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:04.018024 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ttn4\" (UniqueName: \"kubernetes.io/projected/02bbe225-8f61-4fff-af3f-37c464881328-kube-api-access-9ttn4\") pod \"e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8\" (UID: \"02bbe225-8f61-4fff-af3f-37c464881328\") " pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:49:04.018197 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:04.018052 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02bbe225-8f61-4fff-af3f-37c464881328-istiod-ca-cert\") pod \"e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8\" (UID: \"02bbe225-8f61-4fff-af3f-37c464881328\") " pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:49:04.018197 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:04.018104 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02bbe225-8f61-4fff-af3f-37c464881328-istio-data\") pod \"e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8\" (UID: \"02bbe225-8f61-4fff-af3f-37c464881328\") " pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:49:04.018197 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:04.018123 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02bbe225-8f61-4fff-af3f-37c464881328-workload-certs\") pod \"e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8\" (UID: \"02bbe225-8f61-4fff-af3f-37c464881328\") " pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:49:04.018197 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:04.018147 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02bbe225-8f61-4fff-af3f-37c464881328-credential-socket\") pod \"e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8\" (UID: \"02bbe225-8f61-4fff-af3f-37c464881328\") " pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:49:04.018197 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:04.018164 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02bbe225-8f61-4fff-af3f-37c464881328-workload-socket\") pod \"e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8\" (UID: \"02bbe225-8f61-4fff-af3f-37c464881328\") " pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:49:04.018470 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:04.018273 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02bbe225-8f61-4fff-af3f-37c464881328-istio-envoy\") pod \"e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8\" (UID: \"02bbe225-8f61-4fff-af3f-37c464881328\") " pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:49:04.018470 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:04.018318 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02bbe225-8f61-4fff-af3f-37c464881328-istio-podinfo\") pod \"e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8\" (UID: \"02bbe225-8f61-4fff-af3f-37c464881328\") " pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:49:04.093413 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:04.093341 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:49:04.119462 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:04.119438 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02bbe225-8f61-4fff-af3f-37c464881328-workload-certs\") pod \"e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8\" (UID: \"02bbe225-8f61-4fff-af3f-37c464881328\") " pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:49:04.119568 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:04.119470 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02bbe225-8f61-4fff-af3f-37c464881328-credential-socket\") pod \"e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8\" (UID: \"02bbe225-8f61-4fff-af3f-37c464881328\") " pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:49:04.119568 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:04.119489 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02bbe225-8f61-4fff-af3f-37c464881328-workload-socket\") pod \"e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8\" (UID: \"02bbe225-8f61-4fff-af3f-37c464881328\") " pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:49:04.119568 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:04.119523 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02bbe225-8f61-4fff-af3f-37c464881328-istio-envoy\") pod \"e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8\" (UID: \"02bbe225-8f61-4fff-af3f-37c464881328\") " pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:49:04.119568 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:04.119554 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02bbe225-8f61-4fff-af3f-37c464881328-istio-podinfo\") pod \"e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8\" (UID: \"02bbe225-8f61-4fff-af3f-37c464881328\") " pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:49:04.119807 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:04.119574 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02bbe225-8f61-4fff-af3f-37c464881328-istio-token\") pod \"e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8\" (UID: \"02bbe225-8f61-4fff-af3f-37c464881328\") " pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:49:04.119807 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:04.119597 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ttn4\" (UniqueName: \"kubernetes.io/projected/02bbe225-8f61-4fff-af3f-37c464881328-kube-api-access-9ttn4\") pod \"e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8\" (UID: \"02bbe225-8f61-4fff-af3f-37c464881328\") " pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:49:04.119807 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:04.119738 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02bbe225-8f61-4fff-af3f-37c464881328-istiod-ca-cert\") pod \"e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8\" (UID: \"02bbe225-8f61-4fff-af3f-37c464881328\") " pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:49:04.119963 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:04.119817 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02bbe225-8f61-4fff-af3f-37c464881328-istio-data\") pod \"e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8\" (UID: \"02bbe225-8f61-4fff-af3f-37c464881328\") " pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:49:04.119963 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:04.119910 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02bbe225-8f61-4fff-af3f-37c464881328-credential-socket\") pod \"e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8\" (UID: \"02bbe225-8f61-4fff-af3f-37c464881328\") " pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:49:04.120057 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:04.119960 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02bbe225-8f61-4fff-af3f-37c464881328-workload-socket\") pod \"e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8\" (UID: \"02bbe225-8f61-4fff-af3f-37c464881328\") " pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:49:04.120057 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:04.120021 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02bbe225-8f61-4fff-af3f-37c464881328-workload-certs\") pod \"e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8\" (UID: \"02bbe225-8f61-4fff-af3f-37c464881328\") " pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:49:04.120161 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:04.120137 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02bbe225-8f61-4fff-af3f-37c464881328-istio-data\") pod \"e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8\" (UID: \"02bbe225-8f61-4fff-af3f-37c464881328\") " pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:49:04.120482 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:04.120407 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02bbe225-8f61-4fff-af3f-37c464881328-istiod-ca-cert\") pod \"e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8\" (UID: \"02bbe225-8f61-4fff-af3f-37c464881328\") " pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:49:04.122090 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:04.122064 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02bbe225-8f61-4fff-af3f-37c464881328-istio-envoy\") pod \"e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8\" (UID: \"02bbe225-8f61-4fff-af3f-37c464881328\") " pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:49:04.122182 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:04.122168 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02bbe225-8f61-4fff-af3f-37c464881328-istio-podinfo\") pod \"e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8\" (UID: \"02bbe225-8f61-4fff-af3f-37c464881328\") " pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:49:04.127335 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:04.127309 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ttn4\" (UniqueName: \"kubernetes.io/projected/02bbe225-8f61-4fff-af3f-37c464881328-kube-api-access-9ttn4\") pod \"e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8\" (UID: \"02bbe225-8f61-4fff-af3f-37c464881328\") " pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:49:04.127699 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:04.127666 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02bbe225-8f61-4fff-af3f-37c464881328-istio-token\") pod \"e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8\" (UID: \"02bbe225-8f61-4fff-af3f-37c464881328\") " pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:49:04.159389 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:04.159370 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:49:04.433180 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:04.433158 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8"] Sep 08 18:49:04.434786 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:49:04.434762 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02bbe225_8f61_4fff_af3f_37c464881328.slice/crio-0f038eb2e204a5a7e5d99117d03c544c232c670827acb33eb33255032d1031d6 WatchSource:0}: Error finding container 0f038eb2e204a5a7e5d99117d03c544c232c670827acb33eb33255032d1031d6: Status 404 returned error can't find the container with id 0f038eb2e204a5a7e5d99117d03c544c232c670827acb33eb33255032d1031d6 Sep 08 18:49:04.436984 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:04.436953 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 18:49:04.437075 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:04.437018 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 18:49:04.437075 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:04.437046 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 18:49:04.958364 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:04.958329 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" event={"ID":"02bbe225-8f61-4fff-af3f-37c464881328","Type":"ContainerStarted","Data":"6425fbef48b112f480ba96ea95bd97cc768bf885d37adee438f3cb898f486062"} Sep 08 18:49:04.958762 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:04.958377 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" event={"ID":"02bbe225-8f61-4fff-af3f-37c464881328","Type":"ContainerStarted","Data":"0f038eb2e204a5a7e5d99117d03c544c232c670827acb33eb33255032d1031d6"} Sep 08 18:49:04.978907 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:04.978859 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" podStartSLOduration=1.978844049 podStartE2EDuration="1.978844049s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:49:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:49:04.977234372 +0000 UTC m=+1527.183219769" watchObservedRunningTime="2026-09-08 18:49:04.978844049 +0000 UTC m=+1527.184829446" Sep 08 18:49:05.159610 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:05.159586 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:49:05.164529 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:05.164507 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:49:05.964656 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:05.964626 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:49:05.965738 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:05.965718 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:49:06.859503 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:06.859467 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:49:06.959429 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:06.959392 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:49:07.940024 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:07.939994 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:49:08.002617 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:08.002551 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:49:08.103103 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:08.103076 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:49:08.267920 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:08.267852 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:49:08.596775 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:08.596747 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:49:08.743416 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:08.743378 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:49:08.833874 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:08.833840 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-bc23ef-6f4c74f9c8-mb6mq"] Sep 08 18:49:08.837420 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:08.837404 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bc23ef-6f4c74f9c8-mb6mq" Sep 08 18:49:08.839333 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:08.839307 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-bc23ef-dockercfg-xnxtk\"" Sep 08 18:49:08.839458 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:08.839416 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-bc23ef\"" Sep 08 18:49:08.847082 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:08.847028 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-bc23ef-6f4c74f9c8-mb6mq"] Sep 08 18:49:08.861226 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:08.861195 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rn8qf\" (UniqueName: \"kubernetes.io/projected/a3df9ebd-5f33-4306-8647-cf73ecb840c5-kube-api-access-rn8qf\") pod \"payload-pre-processing-e2e-ipp-a-bc23ef-6f4c74f9c8-mb6mq\" (UID: \"a3df9ebd-5f33-4306-8647-cf73ecb840c5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bc23ef-6f4c74f9c8-mb6mq" Sep 08 18:49:08.861392 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:08.861280 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3df9ebd-5f33-4306-8647-cf73ecb840c5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-bc23ef-6f4c74f9c8-mb6mq\" (UID: \"a3df9ebd-5f33-4306-8647-cf73ecb840c5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bc23ef-6f4c74f9c8-mb6mq" Sep 08 18:49:08.903324 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:08.903289 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-bc23ef-7b4dd97bdc-bnmh4"] Sep 08 18:49:08.907616 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:08.907589 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-bc23ef-7b4dd97bdc-bnmh4" Sep 08 18:49:08.916505 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:08.916478 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-bc23ef-7b4dd97bdc-bnmh4"] Sep 08 18:49:08.962254 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:08.962218 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3df9ebd-5f33-4306-8647-cf73ecb840c5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-bc23ef-6f4c74f9c8-mb6mq\" (UID: \"a3df9ebd-5f33-4306-8647-cf73ecb840c5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bc23ef-6f4c74f9c8-mb6mq" Sep 08 18:49:08.962254 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:08.962259 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqfr5\" (UniqueName: \"kubernetes.io/projected/82b01929-a623-4149-a249-9f46fc10c95f-kube-api-access-fqfr5\") pod \"payload-processing-e2e-ipp-a-bc23ef-7b4dd97bdc-bnmh4\" (UID: \"82b01929-a623-4149-a249-9f46fc10c95f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-bc23ef-7b4dd97bdc-bnmh4" Sep 08 18:49:08.962729 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:08.962309 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82b01929-a623-4149-a249-9f46fc10c95f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-bc23ef-7b4dd97bdc-bnmh4\" (UID: \"82b01929-a623-4149-a249-9f46fc10c95f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-bc23ef-7b4dd97bdc-bnmh4" Sep 08 18:49:08.962729 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:08.962394 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rn8qf\" (UniqueName: \"kubernetes.io/projected/a3df9ebd-5f33-4306-8647-cf73ecb840c5-kube-api-access-rn8qf\") pod \"payload-pre-processing-e2e-ipp-a-bc23ef-6f4c74f9c8-mb6mq\" (UID: \"a3df9ebd-5f33-4306-8647-cf73ecb840c5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bc23ef-6f4c74f9c8-mb6mq" Sep 08 18:49:08.962966 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:08.962950 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3df9ebd-5f33-4306-8647-cf73ecb840c5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-bc23ef-6f4c74f9c8-mb6mq\" (UID: \"a3df9ebd-5f33-4306-8647-cf73ecb840c5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bc23ef-6f4c74f9c8-mb6mq" Sep 08 18:49:08.969444 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:08.969420 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rn8qf\" (UniqueName: \"kubernetes.io/projected/a3df9ebd-5f33-4306-8647-cf73ecb840c5-kube-api-access-rn8qf\") pod \"payload-pre-processing-e2e-ipp-a-bc23ef-6f4c74f9c8-mb6mq\" (UID: \"a3df9ebd-5f33-4306-8647-cf73ecb840c5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bc23ef-6f4c74f9c8-mb6mq" Sep 08 18:49:09.062915 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:09.062881 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqfr5\" (UniqueName: \"kubernetes.io/projected/82b01929-a623-4149-a249-9f46fc10c95f-kube-api-access-fqfr5\") pod \"payload-processing-e2e-ipp-a-bc23ef-7b4dd97bdc-bnmh4\" (UID: \"82b01929-a623-4149-a249-9f46fc10c95f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-bc23ef-7b4dd97bdc-bnmh4" Sep 08 18:49:09.063078 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:09.062962 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82b01929-a623-4149-a249-9f46fc10c95f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-bc23ef-7b4dd97bdc-bnmh4\" (UID: \"82b01929-a623-4149-a249-9f46fc10c95f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-bc23ef-7b4dd97bdc-bnmh4" Sep 08 18:49:09.063596 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:09.063577 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82b01929-a623-4149-a249-9f46fc10c95f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-bc23ef-7b4dd97bdc-bnmh4\" (UID: \"82b01929-a623-4149-a249-9f46fc10c95f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-bc23ef-7b4dd97bdc-bnmh4" Sep 08 18:49:09.071718 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:09.071672 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqfr5\" (UniqueName: \"kubernetes.io/projected/82b01929-a623-4149-a249-9f46fc10c95f-kube-api-access-fqfr5\") pod \"payload-processing-e2e-ipp-a-bc23ef-7b4dd97bdc-bnmh4\" (UID: \"82b01929-a623-4149-a249-9f46fc10c95f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-bc23ef-7b4dd97bdc-bnmh4" Sep 08 18:49:09.148844 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:09.148751 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bc23ef-6f4c74f9c8-mb6mq" Sep 08 18:49:09.219065 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:09.219030 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-bc23ef-7b4dd97bdc-bnmh4" Sep 08 18:49:09.422313 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:09.422283 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-bc23ef-6f4c74f9c8-mb6mq"] Sep 08 18:49:09.423034 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:49:09.423009 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3df9ebd_5f33_4306_8647_cf73ecb840c5.slice/crio-c2513dd2527f41b3847af30c230d253552757586d33578fa80dcd8168629d585 WatchSource:0}: Error finding container c2513dd2527f41b3847af30c230d253552757586d33578fa80dcd8168629d585: Status 404 returned error can't find the container with id c2513dd2527f41b3847af30c230d253552757586d33578fa80dcd8168629d585 Sep 08 18:49:09.504959 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:09.504930 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-bc23ef-7b4dd97bdc-bnmh4"] Sep 08 18:49:09.507199 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:49:09.507168 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82b01929_a623_4149_a249_9f46fc10c95f.slice/crio-09272c75e5157f8541d205d41b84a37af719d3de4cf525ecf1126593190f09ff WatchSource:0}: Error finding container 09272c75e5157f8541d205d41b84a37af719d3de4cf525ecf1126593190f09ff: Status 404 returned error can't find the container with id 09272c75e5157f8541d205d41b84a37af719d3de4cf525ecf1126593190f09ff Sep 08 18:49:09.991040 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:09.991007 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-bc23ef-7b4dd97bdc-bnmh4" event={"ID":"82b01929-a623-4149-a249-9f46fc10c95f","Type":"ContainerStarted","Data":"b114b984d88e9a68686938ce7dfc352b6df14017eecb1f641ed5a8a1fea6c790"} Sep 08 18:49:09.991459 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:09.991064 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-bc23ef-7b4dd97bdc-bnmh4" event={"ID":"82b01929-a623-4149-a249-9f46fc10c95f","Type":"ContainerStarted","Data":"09272c75e5157f8541d205d41b84a37af719d3de4cf525ecf1126593190f09ff"} Sep 08 18:49:09.991459 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:09.991223 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-bc23ef-7b4dd97bdc-bnmh4" Sep 08 18:49:09.992822 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:09.992798 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bc23ef-6f4c74f9c8-mb6mq" event={"ID":"a3df9ebd-5f33-4306-8647-cf73ecb840c5","Type":"ContainerStarted","Data":"d8d21ab99791cdedc7e6a69ea80be0ca2b7939a95f8eed281a14a65e7d118534"} Sep 08 18:49:09.992963 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:09.992836 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bc23ef-6f4c74f9c8-mb6mq" event={"ID":"a3df9ebd-5f33-4306-8647-cf73ecb840c5","Type":"ContainerStarted","Data":"c2513dd2527f41b3847af30c230d253552757586d33578fa80dcd8168629d585"} Sep 08 18:49:09.992963 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:09.992872 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bc23ef-6f4c74f9c8-mb6mq" Sep 08 18:49:10.007010 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:10.006958 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-bc23ef-7b4dd97bdc-bnmh4" podStartSLOduration=2.006942598 podStartE2EDuration="2.006942598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:49:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:49:10.005005128 +0000 UTC m=+1532.210990527" watchObservedRunningTime="2026-09-08 18:49:10.006942598 +0000 UTC m=+1532.212927993" Sep 08 18:49:10.021015 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:10.020958 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bc23ef-6f4c74f9c8-mb6mq" podStartSLOduration=2.020942378 podStartE2EDuration="2.020942378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:49:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:49:10.019472171 +0000 UTC m=+1532.225457569" watchObservedRunningTime="2026-09-08 18:49:10.020942378 +0000 UTC m=+1532.226927775" Sep 08 18:49:17.930921 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:17.930880 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:49:21.001526 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:21.001491 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bc23ef-6f4c74f9c8-mb6mq" Sep 08 18:49:21.001979 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:21.001677 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-bc23ef-7b4dd97bdc-bnmh4" Sep 08 18:49:22.005853 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:22.005809 2555 patch_prober.go:28] interesting pod/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 08 18:49:22.006222 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:22.005881 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" podUID="02bbe225-8f61-4fff-af3f-37c464881328" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 08 18:49:23.073974 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:23.073905 2555 generic.go:356] "Generic (PLEG): container finished" podID="02bbe225-8f61-4fff-af3f-37c464881328" containerID="6425fbef48b112f480ba96ea95bd97cc768bf885d37adee438f3cb898f486062" exitCode=0 Sep 08 18:49:23.074335 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:23.073975 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" event={"ID":"02bbe225-8f61-4fff-af3f-37c464881328","Type":"ContainerDied","Data":"6425fbef48b112f480ba96ea95bd97cc768bf885d37adee438f3cb898f486062"} Sep 08 18:49:23.074379 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:23.074365 2555 scope.go:117] "RemoveContainer" containerID="6425fbef48b112f480ba96ea95bd97cc768bf885d37adee438f3cb898f486062" Sep 08 18:49:23.075438 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:23.075409 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 18:49:23.075494 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:23.075477 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 18:49:23.075535 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:23.075524 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 08 18:49:24.082018 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:24.081978 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" event={"ID":"02bbe225-8f61-4fff-af3f-37c464881328","Type":"ContainerStarted","Data":"9b151c37164968085b5651925ae31fd65890636fff3968bd552b8663b0ea8b92"} Sep 08 18:49:24.159992 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:24.159953 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:49:24.160246 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:24.160222 2555 patch_prober.go:28] interesting pod/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body= Sep 08 18:49:24.160329 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:24.160278 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" podUID="02bbe225-8f61-4fff-af3f-37c464881328" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" Sep 08 18:49:25.159900 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:25.159859 2555 patch_prober.go:28] interesting pod/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body= Sep 08 18:49:25.160292 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:25.159924 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" podUID="02bbe225-8f61-4fff-af3f-37c464881328" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" Sep 08 18:49:26.160056 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:26.160021 2555 patch_prober.go:28] interesting pod/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body= Sep 08 18:49:26.160426 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:26.160074 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" podUID="02bbe225-8f61-4fff-af3f-37c464881328" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" Sep 08 18:49:27.160530 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:27.160495 2555 patch_prober.go:28] interesting pod/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body= Sep 08 18:49:27.160905 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:27.160546 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" podUID="02bbe225-8f61-4fff-af3f-37c464881328" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" Sep 08 18:49:28.160226 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:28.160189 2555 patch_prober.go:28] interesting pod/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body= Sep 08 18:49:28.160428 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:28.160253 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" podUID="02bbe225-8f61-4fff-af3f-37c464881328" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" Sep 08 18:49:29.163765 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:29.163738 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:49:29.164238 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:29.164221 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:49:29.164766 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:29.164748 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:49:30.949457 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:30.949423 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:49:31.904274 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:31.904240 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:49:33.504171 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:33.504129 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:49:35.887030 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:35.887004 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:49:35.987796 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:35.987764 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:49:37.153946 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:37.153913 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:49:37.215368 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:37.215332 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:49:37.507122 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:37.507033 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:49:37.604837 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:37.604800 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:49:37.701229 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:37.701194 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:49:37.734825 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:37.734792 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-302afb-5f585bf4c-q9ldr"] Sep 08 18:49:37.739449 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:37.739430 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-302afb-5f585bf4c-q9ldr" Sep 08 18:49:37.744441 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:37.744408 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-302afb\"" Sep 08 18:49:37.744441 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:37.744408 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-302afb-dockercfg-tvnkz\"" Sep 08 18:49:37.755788 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:37.755757 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-302afb-5f585bf4c-q9ldr"] Sep 08 18:49:37.844032 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:37.843999 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cbacc85b-2d7e-42f4-a568-b8b1c95659fe-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-302afb-5f585bf4c-q9ldr\" (UID: \"cbacc85b-2d7e-42f4-a568-b8b1c95659fe\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-302afb-5f585bf4c-q9ldr" Sep 08 18:49:37.844213 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:37.844064 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j46xp\" (UniqueName: \"kubernetes.io/projected/cbacc85b-2d7e-42f4-a568-b8b1c95659fe-kube-api-access-j46xp\") pod \"payload-processing-e2e-ipp-b-302afb-5f585bf4c-q9ldr\" (UID: \"cbacc85b-2d7e-42f4-a568-b8b1c95659fe\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-302afb-5f585bf4c-q9ldr" Sep 08 18:49:37.944973 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:37.944938 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j46xp\" (UniqueName: \"kubernetes.io/projected/cbacc85b-2d7e-42f4-a568-b8b1c95659fe-kube-api-access-j46xp\") pod \"payload-processing-e2e-ipp-b-302afb-5f585bf4c-q9ldr\" (UID: \"cbacc85b-2d7e-42f4-a568-b8b1c95659fe\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-302afb-5f585bf4c-q9ldr" Sep 08 18:49:37.945135 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:37.945093 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cbacc85b-2d7e-42f4-a568-b8b1c95659fe-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-302afb-5f585bf4c-q9ldr\" (UID: \"cbacc85b-2d7e-42f4-a568-b8b1c95659fe\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-302afb-5f585bf4c-q9ldr" Sep 08 18:49:37.945802 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:37.945784 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cbacc85b-2d7e-42f4-a568-b8b1c95659fe-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-302afb-5f585bf4c-q9ldr\" (UID: \"cbacc85b-2d7e-42f4-a568-b8b1c95659fe\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-302afb-5f585bf4c-q9ldr" Sep 08 18:49:37.952336 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:37.952306 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j46xp\" (UniqueName: \"kubernetes.io/projected/cbacc85b-2d7e-42f4-a568-b8b1c95659fe-kube-api-access-j46xp\") pod \"payload-processing-e2e-ipp-b-302afb-5f585bf4c-q9ldr\" (UID: \"cbacc85b-2d7e-42f4-a568-b8b1c95659fe\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-302afb-5f585bf4c-q9ldr" Sep 08 18:49:38.050410 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:38.050374 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-302afb-5f585bf4c-q9ldr" Sep 08 18:49:38.337166 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:38.337132 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-302afb-5f585bf4c-q9ldr"] Sep 08 18:49:39.181523 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:39.181420 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-302afb-5f585bf4c-q9ldr" event={"ID":"cbacc85b-2d7e-42f4-a568-b8b1c95659fe","Type":"ContainerStarted","Data":"ce47701c6a4f65d6939be27665baf5331a73744228d69eb2c38efc77e662df2e"} Sep 08 18:49:39.181523 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:39.181488 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-302afb-5f585bf4c-q9ldr" event={"ID":"cbacc85b-2d7e-42f4-a568-b8b1c95659fe","Type":"ContainerStarted","Data":"8a6ed5f380c0a59cd659637de59299116ce14100e5f5ed2b4c6c0fde02149336"} Sep 08 18:49:39.181523 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:39.181511 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-302afb-5f585bf4c-q9ldr" Sep 08 18:49:39.197601 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:39.197549 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-302afb-5f585bf4c-q9ldr" podStartSLOduration=2.197532998 podStartE2EDuration="2.197532998s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:49:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:49:39.196385746 +0000 UTC m=+1561.402371140" watchObservedRunningTime="2026-09-08 18:49:39.197532998 +0000 UTC m=+1561.403518395" Sep 08 18:49:43.233756 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:43.233717 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:49:50.189659 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:49:50.189622 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-302afb-5f585bf4c-q9ldr" Sep 08 18:50:00.869049 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:50:00.869014 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:50:06.057985 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:50:06.057948 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-302afb-5f585bf4c-q9ldr_cbacc85b-2d7e-42f4-a568-b8b1c95659fe/payload-processing/0.log" Sep 08 18:50:20.238733 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:50:20.238671 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:50:24.039929 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:50:24.039891 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:50:38.811157 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:50:38.811081 2555 scope.go:117] "RemoveContainer" containerID="8419462dee219579546ee58bc7e3b04daf483e225b6e0a8b2c6cd5a8c7085da3" Sep 08 18:50:45.971885 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:50:45.971849 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:50:46.144046 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:50:46.144003 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:50:52.106517 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:50:52.106484 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-bc23ef-7b4dd97bdc-bnmh4_82b01929-a623-4149-a249-9f46fc10c95f/payload-processing/0.log" Sep 08 18:50:52.345926 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:50:52.345896 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-302afb-5f585bf4c-q9ldr_cbacc85b-2d7e-42f4-a568-b8b1c95659fe/payload-processing/0.log" Sep 08 18:50:52.761410 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:50:52.761381 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:50:53.237427 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:50:53.237393 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:50:53.406224 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:50:53.406185 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:50:56.406056 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:50:56.406013 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:50:56.557945 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:50:56.557910 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:50:57.540627 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:50:57.540597 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:50:57.640576 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:50:57.640546 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:50:57.792231 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:50:57.792143 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:50:58.028170 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:50:58.028138 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:50:58.107546 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:50:58.107452 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:50:58.178458 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:50:58.178424 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-61f4b7-c57c975f9-68wf8"] Sep 08 18:50:58.183360 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:50:58.183338 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-61f4b7-c57c975f9-68wf8" Sep 08 18:50:58.185705 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:50:58.185670 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-61f4b7-dockercfg-p7lnb\"" Sep 08 18:50:58.185920 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:50:58.185906 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-61f4b7\"" Sep 08 18:50:58.196907 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:50:58.196876 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-61f4b7-c57c975f9-68wf8"] Sep 08 18:50:58.351745 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:50:58.351675 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a992a9ee-a446-413c-92f0-2450e757ba18-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-61f4b7-c57c975f9-68wf8\" (UID: \"a992a9ee-a446-413c-92f0-2450e757ba18\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-61f4b7-c57c975f9-68wf8" Sep 08 18:50:58.351950 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:50:58.351796 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwwtm\" (UniqueName: \"kubernetes.io/projected/a992a9ee-a446-413c-92f0-2450e757ba18-kube-api-access-wwwtm\") pod \"payload-processing-e2e-ipp-cleanup-61f4b7-c57c975f9-68wf8\" (UID: \"a992a9ee-a446-413c-92f0-2450e757ba18\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-61f4b7-c57c975f9-68wf8" Sep 08 18:50:58.452866 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:50:58.452768 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a992a9ee-a446-413c-92f0-2450e757ba18-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-61f4b7-c57c975f9-68wf8\" (UID: \"a992a9ee-a446-413c-92f0-2450e757ba18\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-61f4b7-c57c975f9-68wf8" Sep 08 18:50:58.452866 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:50:58.452833 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwwtm\" (UniqueName: \"kubernetes.io/projected/a992a9ee-a446-413c-92f0-2450e757ba18-kube-api-access-wwwtm\") pod \"payload-processing-e2e-ipp-cleanup-61f4b7-c57c975f9-68wf8\" (UID: \"a992a9ee-a446-413c-92f0-2450e757ba18\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-61f4b7-c57c975f9-68wf8" Sep 08 18:50:58.453580 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:50:58.453557 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a992a9ee-a446-413c-92f0-2450e757ba18-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-61f4b7-c57c975f9-68wf8\" (UID: \"a992a9ee-a446-413c-92f0-2450e757ba18\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-61f4b7-c57c975f9-68wf8" Sep 08 18:50:58.463612 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:50:58.463574 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwwtm\" (UniqueName: \"kubernetes.io/projected/a992a9ee-a446-413c-92f0-2450e757ba18-kube-api-access-wwwtm\") pod \"payload-processing-e2e-ipp-cleanup-61f4b7-c57c975f9-68wf8\" (UID: \"a992a9ee-a446-413c-92f0-2450e757ba18\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-61f4b7-c57c975f9-68wf8" Sep 08 18:50:58.493470 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:50:58.493428 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-61f4b7-c57c975f9-68wf8" Sep 08 18:50:58.790355 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:50:58.790328 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-61f4b7-c57c975f9-68wf8"] Sep 08 18:50:58.792147 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:50:58.792106 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda992a9ee_a446_413c_92f0_2450e757ba18.slice/crio-e6db46c48a85f1274b5a5447e4e91ee9e124c2b06d28c83dbb7a7d9282f23b5a WatchSource:0}: Error finding container e6db46c48a85f1274b5a5447e4e91ee9e124c2b06d28c83dbb7a7d9282f23b5a: Status 404 returned error can't find the container with id e6db46c48a85f1274b5a5447e4e91ee9e124c2b06d28c83dbb7a7d9282f23b5a Sep 08 18:50:59.661184 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:50:59.661146 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-61f4b7-c57c975f9-68wf8" event={"ID":"a992a9ee-a446-413c-92f0-2450e757ba18","Type":"ContainerStarted","Data":"ac4089e5dd6a9cb98981ba5c5d4f9e7289d32c9dce6330db0789632aa801069a"} Sep 08 18:50:59.661378 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:50:59.661204 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-61f4b7-c57c975f9-68wf8" event={"ID":"a992a9ee-a446-413c-92f0-2450e757ba18","Type":"ContainerStarted","Data":"e6db46c48a85f1274b5a5447e4e91ee9e124c2b06d28c83dbb7a7d9282f23b5a"} Sep 08 18:50:59.661378 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:50:59.661224 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-61f4b7-c57c975f9-68wf8" Sep 08 18:50:59.679883 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:50:59.679833 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-61f4b7-c57c975f9-68wf8" podStartSLOduration=1.6798149150000001 podStartE2EDuration="1.679814915s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:50:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:50:59.677491521 +0000 UTC m=+1641.883476917" watchObservedRunningTime="2026-09-08 18:50:59.679814915 +0000 UTC m=+1641.885800313" Sep 08 18:51:05.740903 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:05.740867 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:51:10.669500 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:10.669470 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-61f4b7-c57c975f9-68wf8" Sep 08 18:51:21.466428 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:21.466390 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:51:28.129020 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:28.128978 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:51:28.159062 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:28.159027 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-61f4b7-c57c975f9-68wf8"] Sep 08 18:51:28.159311 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:28.159278 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-61f4b7-c57c975f9-68wf8" podUID="a992a9ee-a446-413c-92f0-2450e757ba18" containerName="payload-processing" containerID="cri-o://ac4089e5dd6a9cb98981ba5c5d4f9e7289d32c9dce6330db0789632aa801069a" gracePeriod=30 Sep 08 18:51:28.230548 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:28.230512 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:51:28.315825 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:28.315795 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:51:28.387466 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:28.387279 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:51:28.429443 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:28.429416 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-61f4b7-c57c975f9-68wf8" Sep 08 18:51:28.535409 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:28.535372 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wwwtm\" (UniqueName: \"kubernetes.io/projected/a992a9ee-a446-413c-92f0-2450e757ba18-kube-api-access-wwwtm\") pod \"a992a9ee-a446-413c-92f0-2450e757ba18\" (UID: \"a992a9ee-a446-413c-92f0-2450e757ba18\") " Sep 08 18:51:28.535610 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:28.535539 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a992a9ee-a446-413c-92f0-2450e757ba18-plugins-config-volume\") pod \"a992a9ee-a446-413c-92f0-2450e757ba18\" (UID: \"a992a9ee-a446-413c-92f0-2450e757ba18\") " Sep 08 18:51:28.535963 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:28.535938 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a992a9ee-a446-413c-92f0-2450e757ba18-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a992a9ee-a446-413c-92f0-2450e757ba18" (UID: "a992a9ee-a446-413c-92f0-2450e757ba18"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:51:28.537634 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:28.537612 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a992a9ee-a446-413c-92f0-2450e757ba18-kube-api-access-wwwtm" (OuterVolumeSpecName: "kube-api-access-wwwtm") pod "a992a9ee-a446-413c-92f0-2450e757ba18" (UID: "a992a9ee-a446-413c-92f0-2450e757ba18"). InnerVolumeSpecName "kube-api-access-wwwtm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:51:28.636819 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:28.636780 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a992a9ee-a446-413c-92f0-2450e757ba18-plugins-config-volume\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:51:28.636819 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:28.636807 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwwtm\" (UniqueName: \"kubernetes.io/projected/a992a9ee-a446-413c-92f0-2450e757ba18-kube-api-access-wwwtm\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:51:28.752938 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:28.752852 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:51:28.838147 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:28.838109 2555 generic.go:356] "Generic (PLEG): container finished" podID="a992a9ee-a446-413c-92f0-2450e757ba18" containerID="ac4089e5dd6a9cb98981ba5c5d4f9e7289d32c9dce6330db0789632aa801069a" exitCode=0 Sep 08 18:51:28.838329 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:28.838156 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-61f4b7-c57c975f9-68wf8" Sep 08 18:51:28.838329 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:28.838180 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-61f4b7-c57c975f9-68wf8" event={"ID":"a992a9ee-a446-413c-92f0-2450e757ba18","Type":"ContainerDied","Data":"ac4089e5dd6a9cb98981ba5c5d4f9e7289d32c9dce6330db0789632aa801069a"} Sep 08 18:51:28.838329 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:28.838233 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-61f4b7-c57c975f9-68wf8" event={"ID":"a992a9ee-a446-413c-92f0-2450e757ba18","Type":"ContainerDied","Data":"e6db46c48a85f1274b5a5447e4e91ee9e124c2b06d28c83dbb7a7d9282f23b5a"} Sep 08 18:51:28.838329 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:28.838262 2555 scope.go:117] "RemoveContainer" containerID="ac4089e5dd6a9cb98981ba5c5d4f9e7289d32c9dce6330db0789632aa801069a" Sep 08 18:51:28.855795 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:28.855772 2555 scope.go:117] "RemoveContainer" containerID="ac4089e5dd6a9cb98981ba5c5d4f9e7289d32c9dce6330db0789632aa801069a" Sep 08 18:51:28.856150 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:51:28.856131 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac4089e5dd6a9cb98981ba5c5d4f9e7289d32c9dce6330db0789632aa801069a\": container with ID starting with ac4089e5dd6a9cb98981ba5c5d4f9e7289d32c9dce6330db0789632aa801069a not found: ID does not exist" containerID="ac4089e5dd6a9cb98981ba5c5d4f9e7289d32c9dce6330db0789632aa801069a" Sep 08 18:51:28.856267 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:28.856156 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac4089e5dd6a9cb98981ba5c5d4f9e7289d32c9dce6330db0789632aa801069a"} err="failed to get container status \"ac4089e5dd6a9cb98981ba5c5d4f9e7289d32c9dce6330db0789632aa801069a\": rpc error: code = NotFound desc = could not find container \"ac4089e5dd6a9cb98981ba5c5d4f9e7289d32c9dce6330db0789632aa801069a\": container with ID starting with ac4089e5dd6a9cb98981ba5c5d4f9e7289d32c9dce6330db0789632aa801069a not found: ID does not exist" Sep 08 18:51:28.870965 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:28.870930 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-61f4b7-c57c975f9-68wf8"] Sep 08 18:51:28.878268 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:28.878232 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-61f4b7-c57c975f9-68wf8"] Sep 08 18:51:30.325105 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:30.325070 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a992a9ee-a446-413c-92f0-2450e757ba18" path="/var/lib/kubelet/pods/a992a9ee-a446-413c-92f0-2450e757ba18/volumes" Sep 08 18:51:32.586550 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:32.586512 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:51:34.985365 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:34.985331 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:51:35.121819 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:35.121777 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:51:35.456613 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:35.456568 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:51:38.835132 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:38.835101 2555 scope.go:117] "RemoveContainer" containerID="a9a5238a98d6168e5b3c4999538e60e9446450d22b61f6ded9ec9e4cc66a23d6" Sep 08 18:51:39.460572 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:39.460534 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:51:46.393798 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:46.393764 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:51:52.455524 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:52.455485 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:51:54.977625 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:54.977590 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:51:55.056909 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:55.056877 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-bc23ef-7b4dd97bdc-bnmh4"] Sep 08 18:51:55.057150 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:55.057121 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-bc23ef-7b4dd97bdc-bnmh4" podUID="82b01929-a623-4149-a249-9f46fc10c95f" containerName="payload-processing" containerID="cri-o://b114b984d88e9a68686938ce7dfc352b6df14017eecb1f641ed5a8a1fea6c790" gracePeriod=30 Sep 08 18:51:55.078478 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:55.078425 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-bc23ef-6f4c74f9c8-mb6mq"] Sep 08 18:51:55.078743 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:55.078668 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bc23ef-6f4c74f9c8-mb6mq" podUID="a3df9ebd-5f33-4306-8647-cf73ecb840c5" containerName="payload-pre-processing" containerID="cri-o://d8d21ab99791cdedc7e6a69ea80be0ca2b7939a95f8eed281a14a65e7d118534" gracePeriod=30 Sep 08 18:51:55.125045 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:55.125005 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:51:55.281989 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:55.281947 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:51:55.344835 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:55.344811 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-bc23ef-7b4dd97bdc-bnmh4" Sep 08 18:51:55.362871 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:55.362847 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bc23ef-6f4c74f9c8-mb6mq" Sep 08 18:51:55.399434 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:55.399403 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82b01929-a623-4149-a249-9f46fc10c95f-plugins-config-volume\") pod \"82b01929-a623-4149-a249-9f46fc10c95f\" (UID: \"82b01929-a623-4149-a249-9f46fc10c95f\") " Sep 08 18:51:55.399587 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:55.399444 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqfr5\" (UniqueName: \"kubernetes.io/projected/82b01929-a623-4149-a249-9f46fc10c95f-kube-api-access-fqfr5\") pod \"82b01929-a623-4149-a249-9f46fc10c95f\" (UID: \"82b01929-a623-4149-a249-9f46fc10c95f\") " Sep 08 18:51:55.399840 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:55.399812 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82b01929-a623-4149-a249-9f46fc10c95f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "82b01929-a623-4149-a249-9f46fc10c95f" (UID: "82b01929-a623-4149-a249-9f46fc10c95f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:51:55.399951 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:55.399932 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82b01929-a623-4149-a249-9f46fc10c95f-plugins-config-volume\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:51:55.401879 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:55.401859 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82b01929-a623-4149-a249-9f46fc10c95f-kube-api-access-fqfr5" (OuterVolumeSpecName: "kube-api-access-fqfr5") pod "82b01929-a623-4149-a249-9f46fc10c95f" (UID: "82b01929-a623-4149-a249-9f46fc10c95f"). InnerVolumeSpecName "kube-api-access-fqfr5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:51:55.452061 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:55.452020 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:51:55.501201 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:55.501118 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3df9ebd-5f33-4306-8647-cf73ecb840c5-plugins-config-volume\") pod \"a3df9ebd-5f33-4306-8647-cf73ecb840c5\" (UID: \"a3df9ebd-5f33-4306-8647-cf73ecb840c5\") " Sep 08 18:51:55.501350 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:55.501253 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rn8qf\" (UniqueName: \"kubernetes.io/projected/a3df9ebd-5f33-4306-8647-cf73ecb840c5-kube-api-access-rn8qf\") pod \"a3df9ebd-5f33-4306-8647-cf73ecb840c5\" (UID: \"a3df9ebd-5f33-4306-8647-cf73ecb840c5\") " Sep 08 18:51:55.501494 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:55.501482 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqfr5\" (UniqueName: \"kubernetes.io/projected/82b01929-a623-4149-a249-9f46fc10c95f-kube-api-access-fqfr5\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:51:55.501561 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:55.501523 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3df9ebd-5f33-4306-8647-cf73ecb840c5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a3df9ebd-5f33-4306-8647-cf73ecb840c5" (UID: "a3df9ebd-5f33-4306-8647-cf73ecb840c5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:51:55.503431 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:55.503409 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3df9ebd-5f33-4306-8647-cf73ecb840c5-kube-api-access-rn8qf" (OuterVolumeSpecName: "kube-api-access-rn8qf") pod "a3df9ebd-5f33-4306-8647-cf73ecb840c5" (UID: "a3df9ebd-5f33-4306-8647-cf73ecb840c5"). InnerVolumeSpecName "kube-api-access-rn8qf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:51:55.602264 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:55.602220 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rn8qf\" (UniqueName: \"kubernetes.io/projected/a3df9ebd-5f33-4306-8647-cf73ecb840c5-kube-api-access-rn8qf\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:51:55.602264 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:55.602260 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3df9ebd-5f33-4306-8647-cf73ecb840c5-plugins-config-volume\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:51:55.630760 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:55.630726 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:51:55.879962 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:55.879932 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:51:56.003747 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:56.003713 2555 generic.go:356] "Generic (PLEG): container finished" podID="82b01929-a623-4149-a249-9f46fc10c95f" containerID="b114b984d88e9a68686938ce7dfc352b6df14017eecb1f641ed5a8a1fea6c790" exitCode=0 Sep 08 18:51:56.004226 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:56.003767 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-bc23ef-7b4dd97bdc-bnmh4" Sep 08 18:51:56.004226 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:56.003793 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-bc23ef-7b4dd97bdc-bnmh4" event={"ID":"82b01929-a623-4149-a249-9f46fc10c95f","Type":"ContainerDied","Data":"b114b984d88e9a68686938ce7dfc352b6df14017eecb1f641ed5a8a1fea6c790"} Sep 08 18:51:56.004226 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:56.003853 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-bc23ef-7b4dd97bdc-bnmh4" event={"ID":"82b01929-a623-4149-a249-9f46fc10c95f","Type":"ContainerDied","Data":"09272c75e5157f8541d205d41b84a37af719d3de4cf525ecf1126593190f09ff"} Sep 08 18:51:56.004226 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:56.003878 2555 scope.go:117] "RemoveContainer" containerID="b114b984d88e9a68686938ce7dfc352b6df14017eecb1f641ed5a8a1fea6c790" Sep 08 18:51:56.005225 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:56.005192 2555 generic.go:356] "Generic (PLEG): container finished" podID="a3df9ebd-5f33-4306-8647-cf73ecb840c5" containerID="d8d21ab99791cdedc7e6a69ea80be0ca2b7939a95f8eed281a14a65e7d118534" exitCode=0 Sep 08 18:51:56.005383 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:56.005232 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bc23ef-6f4c74f9c8-mb6mq" event={"ID":"a3df9ebd-5f33-4306-8647-cf73ecb840c5","Type":"ContainerDied","Data":"d8d21ab99791cdedc7e6a69ea80be0ca2b7939a95f8eed281a14a65e7d118534"} Sep 08 18:51:56.005383 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:56.005259 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bc23ef-6f4c74f9c8-mb6mq" Sep 08 18:51:56.005383 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:56.005284 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-bc23ef-6f4c74f9c8-mb6mq" event={"ID":"a3df9ebd-5f33-4306-8647-cf73ecb840c5","Type":"ContainerDied","Data":"c2513dd2527f41b3847af30c230d253552757586d33578fa80dcd8168629d585"} Sep 08 18:51:56.015243 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:56.015216 2555 scope.go:117] "RemoveContainer" containerID="b114b984d88e9a68686938ce7dfc352b6df14017eecb1f641ed5a8a1fea6c790" Sep 08 18:51:56.016281 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:51:56.016228 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b114b984d88e9a68686938ce7dfc352b6df14017eecb1f641ed5a8a1fea6c790\": container with ID starting with b114b984d88e9a68686938ce7dfc352b6df14017eecb1f641ed5a8a1fea6c790 not found: ID does not exist" containerID="b114b984d88e9a68686938ce7dfc352b6df14017eecb1f641ed5a8a1fea6c790" Sep 08 18:51:56.016398 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:56.016274 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b114b984d88e9a68686938ce7dfc352b6df14017eecb1f641ed5a8a1fea6c790"} err="failed to get container status \"b114b984d88e9a68686938ce7dfc352b6df14017eecb1f641ed5a8a1fea6c790\": rpc error: code = NotFound desc = could not find container \"b114b984d88e9a68686938ce7dfc352b6df14017eecb1f641ed5a8a1fea6c790\": container with ID starting with b114b984d88e9a68686938ce7dfc352b6df14017eecb1f641ed5a8a1fea6c790 not found: ID does not exist" Sep 08 18:51:56.016398 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:56.016312 2555 scope.go:117] "RemoveContainer" containerID="d8d21ab99791cdedc7e6a69ea80be0ca2b7939a95f8eed281a14a65e7d118534" Sep 08 18:51:56.027941 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:56.027922 2555 scope.go:117] "RemoveContainer" containerID="d8d21ab99791cdedc7e6a69ea80be0ca2b7939a95f8eed281a14a65e7d118534" Sep 08 18:51:56.028259 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:51:56.028239 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8d21ab99791cdedc7e6a69ea80be0ca2b7939a95f8eed281a14a65e7d118534\": container with ID starting with d8d21ab99791cdedc7e6a69ea80be0ca2b7939a95f8eed281a14a65e7d118534 not found: ID does not exist" containerID="d8d21ab99791cdedc7e6a69ea80be0ca2b7939a95f8eed281a14a65e7d118534" Sep 08 18:51:56.028306 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:56.028269 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8d21ab99791cdedc7e6a69ea80be0ca2b7939a95f8eed281a14a65e7d118534"} err="failed to get container status \"d8d21ab99791cdedc7e6a69ea80be0ca2b7939a95f8eed281a14a65e7d118534\": rpc error: code = NotFound desc = could not find container \"d8d21ab99791cdedc7e6a69ea80be0ca2b7939a95f8eed281a14a65e7d118534\": container with ID starting with d8d21ab99791cdedc7e6a69ea80be0ca2b7939a95f8eed281a14a65e7d118534 not found: ID does not exist" Sep 08 18:51:56.045914 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:56.045879 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-bc23ef-6f4c74f9c8-mb6mq"] Sep 08 18:51:56.049881 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:56.049852 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-bc23ef-6f4c74f9c8-mb6mq"] Sep 08 18:51:56.068012 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:56.067800 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-bc23ef-7b4dd97bdc-bnmh4"] Sep 08 18:51:56.072512 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:56.072475 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-bc23ef-7b4dd97bdc-bnmh4"] Sep 08 18:51:56.324887 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:56.324856 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82b01929-a623-4149-a249-9f46fc10c95f" path="/var/lib/kubelet/pods/82b01929-a623-4149-a249-9f46fc10c95f/volumes" Sep 08 18:51:56.325255 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:56.325241 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3df9ebd-5f33-4306-8647-cf73ecb840c5" path="/var/lib/kubelet/pods/a3df9ebd-5f33-4306-8647-cf73ecb840c5/volumes" Sep 08 18:51:58.579223 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:51:58.579186 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:52:00.885757 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:52:00.885633 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:52:04.632366 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:52:04.632330 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:52:22.207489 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:52:22.207452 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:52:37.719956 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:52:37.719911 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:52:38.046635 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:52:38.046604 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:52:43.520105 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:52:43.520054 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:52:50.823266 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:52:50.823229 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:52:58.088627 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:52:58.088591 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:52:58.156629 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:52:58.156595 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:52:58.334958 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:52:58.334928 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8"] Sep 08 18:52:58.335190 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:52:58.335157 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" podUID="02bbe225-8f61-4fff-af3f-37c464881328" containerName="istio-proxy" containerID="cri-o://9b151c37164968085b5651925ae31fd65890636fff3968bd552b8663b0ea8b92" gracePeriod=30 Sep 08 18:52:58.359609 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:52:58.359537 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:52:58.845965 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:52:58.845932 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:52:59.166739 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:52:59.166638 2555 patch_prober.go:28] interesting pod/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 08 18:52:59.166739 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:52:59.166725 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" podUID="02bbe225-8f61-4fff-af3f-37c464881328" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 08 18:53:03.601813 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:03.601755 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:53:03.639483 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:03.639448 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02bbe225-8f61-4fff-af3f-37c464881328-workload-socket\") pod \"02bbe225-8f61-4fff-af3f-37c464881328\" (UID: \"02bbe225-8f61-4fff-af3f-37c464881328\") " Sep 08 18:53:03.639630 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:03.639489 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9ttn4\" (UniqueName: \"kubernetes.io/projected/02bbe225-8f61-4fff-af3f-37c464881328-kube-api-access-9ttn4\") pod \"02bbe225-8f61-4fff-af3f-37c464881328\" (UID: \"02bbe225-8f61-4fff-af3f-37c464881328\") " Sep 08 18:53:03.639630 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:03.639514 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02bbe225-8f61-4fff-af3f-37c464881328-workload-certs\") pod \"02bbe225-8f61-4fff-af3f-37c464881328\" (UID: \"02bbe225-8f61-4fff-af3f-37c464881328\") " Sep 08 18:53:03.639630 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:03.639547 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02bbe225-8f61-4fff-af3f-37c464881328-istio-envoy\") pod \"02bbe225-8f61-4fff-af3f-37c464881328\" (UID: \"02bbe225-8f61-4fff-af3f-37c464881328\") " Sep 08 18:53:03.639630 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:03.639603 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02bbe225-8f61-4fff-af3f-37c464881328-istio-podinfo\") pod \"02bbe225-8f61-4fff-af3f-37c464881328\" (UID: \"02bbe225-8f61-4fff-af3f-37c464881328\") " Sep 08 18:53:03.639876 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:03.639638 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02bbe225-8f61-4fff-af3f-37c464881328-istio-data\") pod \"02bbe225-8f61-4fff-af3f-37c464881328\" (UID: \"02bbe225-8f61-4fff-af3f-37c464881328\") " Sep 08 18:53:03.639876 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:03.639659 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02bbe225-8f61-4fff-af3f-37c464881328-credential-socket\") pod \"02bbe225-8f61-4fff-af3f-37c464881328\" (UID: \"02bbe225-8f61-4fff-af3f-37c464881328\") " Sep 08 18:53:03.639876 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:03.639749 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02bbe225-8f61-4fff-af3f-37c464881328-istiod-ca-cert\") pod \"02bbe225-8f61-4fff-af3f-37c464881328\" (UID: \"02bbe225-8f61-4fff-af3f-37c464881328\") " Sep 08 18:53:03.639876 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:03.639808 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02bbe225-8f61-4fff-af3f-37c464881328-istio-token\") pod \"02bbe225-8f61-4fff-af3f-37c464881328\" (UID: \"02bbe225-8f61-4fff-af3f-37c464881328\") " Sep 08 18:53:03.639876 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:03.639782 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02bbe225-8f61-4fff-af3f-37c464881328-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "02bbe225-8f61-4fff-af3f-37c464881328" (UID: "02bbe225-8f61-4fff-af3f-37c464881328"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:53:03.639876 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:03.639846 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02bbe225-8f61-4fff-af3f-37c464881328-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "02bbe225-8f61-4fff-af3f-37c464881328" (UID: "02bbe225-8f61-4fff-af3f-37c464881328"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:53:03.640169 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:03.639970 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02bbe225-8f61-4fff-af3f-37c464881328-istio-data" (OuterVolumeSpecName: "istio-data") pod "02bbe225-8f61-4fff-af3f-37c464881328" (UID: "02bbe225-8f61-4fff-af3f-37c464881328"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:53:03.640169 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:03.640140 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02bbe225-8f61-4fff-af3f-37c464881328-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "02bbe225-8f61-4fff-af3f-37c464881328" (UID: "02bbe225-8f61-4fff-af3f-37c464881328"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:53:03.640169 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:03.640158 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02bbe225-8f61-4fff-af3f-37c464881328-workload-socket\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:53:03.640348 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:03.640179 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02bbe225-8f61-4fff-af3f-37c464881328-workload-certs\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:53:03.640348 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:03.640192 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02bbe225-8f61-4fff-af3f-37c464881328-istio-data\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:53:03.640580 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:03.640560 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02bbe225-8f61-4fff-af3f-37c464881328-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "02bbe225-8f61-4fff-af3f-37c464881328" (UID: "02bbe225-8f61-4fff-af3f-37c464881328"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:53:03.642111 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:03.642086 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02bbe225-8f61-4fff-af3f-37c464881328-istio-token" (OuterVolumeSpecName: "istio-token") pod "02bbe225-8f61-4fff-af3f-37c464881328" (UID: "02bbe225-8f61-4fff-af3f-37c464881328"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:53:03.642401 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:03.642383 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02bbe225-8f61-4fff-af3f-37c464881328-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "02bbe225-8f61-4fff-af3f-37c464881328" (UID: "02bbe225-8f61-4fff-af3f-37c464881328"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:53:03.642472 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:03.642451 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02bbe225-8f61-4fff-af3f-37c464881328-kube-api-access-9ttn4" (OuterVolumeSpecName: "kube-api-access-9ttn4") pod "02bbe225-8f61-4fff-af3f-37c464881328" (UID: "02bbe225-8f61-4fff-af3f-37c464881328"). InnerVolumeSpecName "kube-api-access-9ttn4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:53:03.642534 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:03.642478 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/02bbe225-8f61-4fff-af3f-37c464881328-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "02bbe225-8f61-4fff-af3f-37c464881328" (UID: "02bbe225-8f61-4fff-af3f-37c464881328"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 08 18:53:03.741226 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:03.741195 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ttn4\" (UniqueName: \"kubernetes.io/projected/02bbe225-8f61-4fff-af3f-37c464881328-kube-api-access-9ttn4\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:53:03.741226 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:03.741222 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02bbe225-8f61-4fff-af3f-37c464881328-istio-envoy\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:53:03.741226 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:03.741231 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02bbe225-8f61-4fff-af3f-37c464881328-istio-podinfo\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:53:03.741460 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:03.741239 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02bbe225-8f61-4fff-af3f-37c464881328-credential-socket\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:53:03.741460 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:03.741247 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02bbe225-8f61-4fff-af3f-37c464881328-istiod-ca-cert\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:53:03.741460 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:03.741255 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02bbe225-8f61-4fff-af3f-37c464881328-istio-token\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:53:04.404126 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:04.404090 2555 generic.go:356] "Generic (PLEG): container finished" podID="02bbe225-8f61-4fff-af3f-37c464881328" containerID="9b151c37164968085b5651925ae31fd65890636fff3968bd552b8663b0ea8b92" exitCode=0 Sep 08 18:53:04.404275 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:04.404145 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" Sep 08 18:53:04.404275 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:04.404165 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" event={"ID":"02bbe225-8f61-4fff-af3f-37c464881328","Type":"ContainerDied","Data":"9b151c37164968085b5651925ae31fd65890636fff3968bd552b8663b0ea8b92"} Sep 08 18:53:04.404275 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:04.404225 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8" event={"ID":"02bbe225-8f61-4fff-af3f-37c464881328","Type":"ContainerDied","Data":"0f038eb2e204a5a7e5d99117d03c544c232c670827acb33eb33255032d1031d6"} Sep 08 18:53:04.404275 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:04.404248 2555 scope.go:117] "RemoveContainer" containerID="9b151c37164968085b5651925ae31fd65890636fff3968bd552b8663b0ea8b92" Sep 08 18:53:04.414159 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:04.414136 2555 scope.go:117] "RemoveContainer" containerID="6425fbef48b112f480ba96ea95bd97cc768bf885d37adee438f3cb898f486062" Sep 08 18:53:04.421826 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:04.421800 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8"] Sep 08 18:53:04.423416 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:04.423396 2555 scope.go:117] "RemoveContainer" containerID="9b151c37164968085b5651925ae31fd65890636fff3968bd552b8663b0ea8b92" Sep 08 18:53:04.423782 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:53:04.423762 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b151c37164968085b5651925ae31fd65890636fff3968bd552b8663b0ea8b92\": container with ID starting with 9b151c37164968085b5651925ae31fd65890636fff3968bd552b8663b0ea8b92 not found: ID does not exist" containerID="9b151c37164968085b5651925ae31fd65890636fff3968bd552b8663b0ea8b92" Sep 08 18:53:04.423876 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:04.423794 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b151c37164968085b5651925ae31fd65890636fff3968bd552b8663b0ea8b92"} err="failed to get container status \"9b151c37164968085b5651925ae31fd65890636fff3968bd552b8663b0ea8b92\": rpc error: code = NotFound desc = could not find container \"9b151c37164968085b5651925ae31fd65890636fff3968bd552b8663b0ea8b92\": container with ID starting with 9b151c37164968085b5651925ae31fd65890636fff3968bd552b8663b0ea8b92 not found: ID does not exist" Sep 08 18:53:04.423876 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:04.423820 2555 scope.go:117] "RemoveContainer" containerID="6425fbef48b112f480ba96ea95bd97cc768bf885d37adee438f3cb898f486062" Sep 08 18:53:04.424129 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:53:04.424106 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6425fbef48b112f480ba96ea95bd97cc768bf885d37adee438f3cb898f486062\": container with ID starting with 6425fbef48b112f480ba96ea95bd97cc768bf885d37adee438f3cb898f486062 not found: ID does not exist" containerID="6425fbef48b112f480ba96ea95bd97cc768bf885d37adee438f3cb898f486062" Sep 08 18:53:04.424182 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:04.424139 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6425fbef48b112f480ba96ea95bd97cc768bf885d37adee438f3cb898f486062"} err="failed to get container status \"6425fbef48b112f480ba96ea95bd97cc768bf885d37adee438f3cb898f486062\": rpc error: code = NotFound desc = could not find container \"6425fbef48b112f480ba96ea95bd97cc768bf885d37adee438f3cb898f486062\": container with ID starting with 6425fbef48b112f480ba96ea95bd97cc768bf885d37adee438f3cb898f486062 not found: ID does not exist" Sep 08 18:53:04.425455 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:04.425438 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-bc23ef-openshift-default-7fc4c5cc4d-vn9k8"] Sep 08 18:53:06.324357 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:06.324322 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02bbe225-8f61-4fff-af3f-37c464881328" path="/var/lib/kubelet/pods/02bbe225-8f61-4fff-af3f-37c464881328/volumes" Sep 08 18:53:09.350871 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:09.350833 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:53:09.383180 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:09.383140 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-302afb-5f585bf4c-q9ldr"] Sep 08 18:53:09.383424 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:09.383395 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-302afb-5f585bf4c-q9ldr" podUID="cbacc85b-2d7e-42f4-a568-b8b1c95659fe" containerName="payload-processing" containerID="cri-o://ce47701c6a4f65d6939be27665baf5331a73744228d69eb2c38efc77e662df2e" gracePeriod=30 Sep 08 18:53:09.450757 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:09.450720 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:53:09.542604 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:09.542571 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:53:09.616024 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:09.615949 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:53:09.641463 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:09.641439 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-302afb-5f585bf4c-q9ldr" Sep 08 18:53:09.695860 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:09.695835 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j46xp\" (UniqueName: \"kubernetes.io/projected/cbacc85b-2d7e-42f4-a568-b8b1c95659fe-kube-api-access-j46xp\") pod \"cbacc85b-2d7e-42f4-a568-b8b1c95659fe\" (UID: \"cbacc85b-2d7e-42f4-a568-b8b1c95659fe\") " Sep 08 18:53:09.696005 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:09.695888 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cbacc85b-2d7e-42f4-a568-b8b1c95659fe-plugins-config-volume\") pod \"cbacc85b-2d7e-42f4-a568-b8b1c95659fe\" (UID: \"cbacc85b-2d7e-42f4-a568-b8b1c95659fe\") " Sep 08 18:53:09.696271 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:09.696246 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbacc85b-2d7e-42f4-a568-b8b1c95659fe-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cbacc85b-2d7e-42f4-a568-b8b1c95659fe" (UID: "cbacc85b-2d7e-42f4-a568-b8b1c95659fe"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:53:09.698105 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:09.698087 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbacc85b-2d7e-42f4-a568-b8b1c95659fe-kube-api-access-j46xp" (OuterVolumeSpecName: "kube-api-access-j46xp") pod "cbacc85b-2d7e-42f4-a568-b8b1c95659fe" (UID: "cbacc85b-2d7e-42f4-a568-b8b1c95659fe"). InnerVolumeSpecName "kube-api-access-j46xp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:53:09.797151 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:09.797123 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j46xp\" (UniqueName: \"kubernetes.io/projected/cbacc85b-2d7e-42f4-a568-b8b1c95659fe-kube-api-access-j46xp\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:53:09.797151 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:09.797147 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cbacc85b-2d7e-42f4-a568-b8b1c95659fe-plugins-config-volume\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:53:09.904323 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:09.904253 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:53:10.439995 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:10.439962 2555 generic.go:356] "Generic (PLEG): container finished" podID="cbacc85b-2d7e-42f4-a568-b8b1c95659fe" containerID="ce47701c6a4f65d6939be27665baf5331a73744228d69eb2c38efc77e662df2e" exitCode=0 Sep 08 18:53:10.440397 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:10.440011 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-302afb-5f585bf4c-q9ldr" Sep 08 18:53:10.440397 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:10.440046 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-302afb-5f585bf4c-q9ldr" event={"ID":"cbacc85b-2d7e-42f4-a568-b8b1c95659fe","Type":"ContainerDied","Data":"ce47701c6a4f65d6939be27665baf5331a73744228d69eb2c38efc77e662df2e"} Sep 08 18:53:10.440397 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:10.440098 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-302afb-5f585bf4c-q9ldr" event={"ID":"cbacc85b-2d7e-42f4-a568-b8b1c95659fe","Type":"ContainerDied","Data":"8a6ed5f380c0a59cd659637de59299116ce14100e5f5ed2b4c6c0fde02149336"} Sep 08 18:53:10.440397 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:10.440124 2555 scope.go:117] "RemoveContainer" containerID="ce47701c6a4f65d6939be27665baf5331a73744228d69eb2c38efc77e662df2e" Sep 08 18:53:10.449286 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:10.449269 2555 scope.go:117] "RemoveContainer" containerID="ce47701c6a4f65d6939be27665baf5331a73744228d69eb2c38efc77e662df2e" Sep 08 18:53:10.449569 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:53:10.449550 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce47701c6a4f65d6939be27665baf5331a73744228d69eb2c38efc77e662df2e\": container with ID starting with ce47701c6a4f65d6939be27665baf5331a73744228d69eb2c38efc77e662df2e not found: ID does not exist" containerID="ce47701c6a4f65d6939be27665baf5331a73744228d69eb2c38efc77e662df2e" Sep 08 18:53:10.449620 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:10.449582 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce47701c6a4f65d6939be27665baf5331a73744228d69eb2c38efc77e662df2e"} err="failed to get container status \"ce47701c6a4f65d6939be27665baf5331a73744228d69eb2c38efc77e662df2e\": rpc error: code = NotFound desc = could not find container \"ce47701c6a4f65d6939be27665baf5331a73744228d69eb2c38efc77e662df2e\": container with ID starting with ce47701c6a4f65d6939be27665baf5331a73744228d69eb2c38efc77e662df2e not found: ID does not exist" Sep 08 18:53:10.460358 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:10.460339 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-302afb-5f585bf4c-q9ldr"] Sep 08 18:53:10.464486 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:10.464465 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-302afb-5f585bf4c-q9ldr"] Sep 08 18:53:12.324943 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:12.324912 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbacc85b-2d7e-42f4-a568-b8b1c95659fe" path="/var/lib/kubelet/pods/cbacc85b-2d7e-42f4-a568-b8b1c95659fe/volumes" Sep 08 18:53:12.527889 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:12.527854 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:53:15.720448 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:15.720415 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:53:15.817192 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:15.817150 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:53:15.986257 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:15.986160 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:53:16.325570 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:16.325528 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-792c35-fr4lh"] Sep 08 18:53:16.325955 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:16.325940 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02bbe225-8f61-4fff-af3f-37c464881328" containerName="istio-proxy" Sep 08 18:53:16.326007 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:16.325957 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="02bbe225-8f61-4fff-af3f-37c464881328" containerName="istio-proxy" Sep 08 18:53:16.326007 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:16.325976 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02bbe225-8f61-4fff-af3f-37c464881328" containerName="istio-proxy" Sep 08 18:53:16.326007 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:16.325982 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="02bbe225-8f61-4fff-af3f-37c464881328" containerName="istio-proxy" Sep 08 18:53:16.326007 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:16.325988 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbacc85b-2d7e-42f4-a568-b8b1c95659fe" containerName="payload-processing" Sep 08 18:53:16.326007 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:16.325994 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbacc85b-2d7e-42f4-a568-b8b1c95659fe" containerName="payload-processing" Sep 08 18:53:16.326007 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:16.326000 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a992a9ee-a446-413c-92f0-2450e757ba18" containerName="payload-processing" Sep 08 18:53:16.326007 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:16.326005 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="a992a9ee-a446-413c-92f0-2450e757ba18" containerName="payload-processing" Sep 08 18:53:16.326226 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:16.326013 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3df9ebd-5f33-4306-8647-cf73ecb840c5" containerName="payload-pre-processing" Sep 08 18:53:16.326226 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:16.326018 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3df9ebd-5f33-4306-8647-cf73ecb840c5" containerName="payload-pre-processing" Sep 08 18:53:16.326226 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:16.326029 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82b01929-a623-4149-a249-9f46fc10c95f" containerName="payload-processing" Sep 08 18:53:16.326226 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:16.326034 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="82b01929-a623-4149-a249-9f46fc10c95f" containerName="payload-processing" Sep 08 18:53:16.326226 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:16.326091 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3df9ebd-5f33-4306-8647-cf73ecb840c5" containerName="payload-pre-processing" Sep 08 18:53:16.326226 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:16.326099 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="82b01929-a623-4149-a249-9f46fc10c95f" containerName="payload-processing" Sep 08 18:53:16.326226 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:16.326105 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="02bbe225-8f61-4fff-af3f-37c464881328" containerName="istio-proxy" Sep 08 18:53:16.326226 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:16.326110 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="02bbe225-8f61-4fff-af3f-37c464881328" containerName="istio-proxy" Sep 08 18:53:16.326226 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:16.326115 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbacc85b-2d7e-42f4-a568-b8b1c95659fe" containerName="payload-processing" Sep 08 18:53:16.326226 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:16.326122 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="a992a9ee-a446-413c-92f0-2450e757ba18" containerName="payload-processing" Sep 08 18:53:16.330360 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:16.330346 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-792c35-fr4lh" Sep 08 18:53:16.332559 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:16.332532 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 08 18:53:16.332724 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:16.332563 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-g6q8h\"" Sep 08 18:53:16.335656 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:16.335632 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-792c35-fr4lh"] Sep 08 18:53:16.447501 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:16.447462 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:53:16.454795 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:16.454670 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed5d375f-bf6f-4657-b9d3-5be0d6874942-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-792c35-fr4lh\" (UID: \"ed5d375f-bf6f-4657-b9d3-5be0d6874942\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-792c35-fr4lh" Sep 08 18:53:16.556015 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:16.555982 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed5d375f-bf6f-4657-b9d3-5be0d6874942-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-792c35-fr4lh\" (UID: \"ed5d375f-bf6f-4657-b9d3-5be0d6874942\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-792c35-fr4lh" Sep 08 18:53:16.556596 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:16.556577 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed5d375f-bf6f-4657-b9d3-5be0d6874942-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-792c35-fr4lh\" (UID: \"ed5d375f-bf6f-4657-b9d3-5be0d6874942\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-792c35-fr4lh" Sep 08 18:53:16.641490 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:16.641467 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-792c35-fr4lh" Sep 08 18:53:16.912905 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:16.912879 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-792c35-fr4lh"] Sep 08 18:53:16.914507 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:53:16.914481 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded5d375f_bf6f_4657_b9d3_5be0d6874942.slice/crio-3ebce012b3516908118c11075545e3650975c99e9855445a34b03304dead7b52 WatchSource:0}: Error finding container 3ebce012b3516908118c11075545e3650975c99e9855445a34b03304dead7b52: Status 404 returned error can't find the container with id 3ebce012b3516908118c11075545e3650975c99e9855445a34b03304dead7b52 Sep 08 18:53:17.484064 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:17.483972 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-792c35-fr4lh" event={"ID":"ed5d375f-bf6f-4657-b9d3-5be0d6874942","Type":"ContainerStarted","Data":"82a44a81f6d144dd704b5f5f73ef7ab6dde81c3c147b54b4a56d580365f9f3c3"} Sep 08 18:53:17.484064 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:17.484024 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-792c35-fr4lh" event={"ID":"ed5d375f-bf6f-4657-b9d3-5be0d6874942","Type":"ContainerStarted","Data":"3ebce012b3516908118c11075545e3650975c99e9855445a34b03304dead7b52"} Sep 08 18:53:17.499465 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:17.499418 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-792c35-fr4lh" podStartSLOduration=1.4994019 podStartE2EDuration="1.4994019s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:53:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:53:17.497570476 +0000 UTC m=+1779.703555875" watchObservedRunningTime="2026-09-08 18:53:17.4994019 +0000 UTC m=+1779.705387342" Sep 08 18:53:18.490605 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:18.490570 2555 generic.go:356] "Generic (PLEG): container finished" podID="ed5d375f-bf6f-4657-b9d3-5be0d6874942" containerID="82a44a81f6d144dd704b5f5f73ef7ab6dde81c3c147b54b4a56d580365f9f3c3" exitCode=0 Sep 08 18:53:18.491039 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:18.490664 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-792c35-fr4lh" event={"ID":"ed5d375f-bf6f-4657-b9d3-5be0d6874942","Type":"ContainerDied","Data":"82a44a81f6d144dd704b5f5f73ef7ab6dde81c3c147b54b4a56d580365f9f3c3"} Sep 08 18:53:19.628076 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:19.628045 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-792c35-fr4lh" Sep 08 18:53:19.683520 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:19.683487 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed5d375f-bf6f-4657-b9d3-5be0d6874942-maas-api-service-ca\") pod \"ed5d375f-bf6f-4657-b9d3-5be0d6874942\" (UID: \"ed5d375f-bf6f-4657-b9d3-5be0d6874942\") " Sep 08 18:53:19.683881 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:19.683855 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed5d375f-bf6f-4657-b9d3-5be0d6874942-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ed5d375f-bf6f-4657-b9d3-5be0d6874942" (UID: "ed5d375f-bf6f-4657-b9d3-5be0d6874942"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:53:19.785189 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:19.785097 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed5d375f-bf6f-4657-b9d3-5be0d6874942-maas-api-service-ca\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:53:20.504301 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:20.504270 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-792c35-fr4lh" event={"ID":"ed5d375f-bf6f-4657-b9d3-5be0d6874942","Type":"ContainerDied","Data":"3ebce012b3516908118c11075545e3650975c99e9855445a34b03304dead7b52"} Sep 08 18:53:20.504301 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:20.504300 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-792c35-fr4lh" Sep 08 18:53:20.504500 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:20.504321 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3ebce012b3516908118c11075545e3650975c99e9855445a34b03304dead7b52" Sep 08 18:53:26.741791 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:26.741753 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:53:26.742674 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:26.742648 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-792c35-29814885-ttblp"] Sep 08 18:53:26.745007 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:26.744982 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-792c35-29814885-ttblp"] Sep 08 18:53:26.810799 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:26.810769 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:53:26.911799 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:26.911769 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:53:26.924700 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:26.924655 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-792c35-fr4lh"] Sep 08 18:53:26.927040 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:26.927022 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-792c35-fr4lh"] Sep 08 18:53:27.004274 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:27.004157 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:53:27.317592 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:27.317545 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:53:28.324443 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:28.324412 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed5d375f-bf6f-4657-b9d3-5be0d6874942" path="/var/lib/kubelet/pods/ed5d375f-bf6f-4657-b9d3-5be0d6874942/volumes" Sep 08 18:53:28.324833 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:28.324707 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4eff982-06b0-410a-887d-b4d4cb85336e" path="/var/lib/kubelet/pods/f4eff982-06b0-410a-887d-b4d4cb85336e/volumes" Sep 08 18:53:29.727030 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:29.726952 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:53:33.509482 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:33.509444 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:53:33.576152 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:33.576119 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:53:33.767271 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:33.767190 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:53:34.245727 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:34.245676 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:53:36.379378 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:36.379342 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:53:36.433171 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:36.433137 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:53:38.477706 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:38.477662 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9n28f_911c61b8-766b-4ae8-b046-784e22e85889/console-operator/1.log" Sep 08 18:53:38.487356 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:38.487325 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9n28f_911c61b8-766b-4ae8-b046-784e22e85889/console-operator/1.log" Sep 08 18:53:38.881375 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:38.881342 2555 scope.go:117] "RemoveContainer" containerID="c865b21c6bc7f255ab17f662a28fb7829cf1e29aeb04ebe9444de8394d444564" Sep 08 18:53:44.622073 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:44.622037 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6255f2-29814885-7g47b"] Sep 08 18:53:44.624394 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:44.624373 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6255f2-29814885-7g47b"] Sep 08 18:53:44.654901 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:44.654865 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6255f2-5b689d4b8b-j58cg"] Sep 08 18:53:44.655167 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:44.655100 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6255f2-5b689d4b8b-j58cg" podUID="72a1a9df-dd32-4a49-a5b5-72a4ce01b798" containerName="payload-pre-processing" containerID="cri-o://5737fe8def1fc2333a2cc65c112a0f7d51bf3ec31ef272fda43713c7a9b48f53" gracePeriod=30 Sep 08 18:53:44.682736 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:44.682673 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:53:44.782549 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:44.782523 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:53:44.837436 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:44.837410 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:53:44.916793 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:44.916768 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6255f2-5b689d4b8b-j58cg" Sep 08 18:53:45.017477 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:45.017447 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tpdq6\" (UniqueName: \"kubernetes.io/projected/72a1a9df-dd32-4a49-a5b5-72a4ce01b798-kube-api-access-tpdq6\") pod \"72a1a9df-dd32-4a49-a5b5-72a4ce01b798\" (UID: \"72a1a9df-dd32-4a49-a5b5-72a4ce01b798\") " Sep 08 18:53:45.017633 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:45.017578 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72a1a9df-dd32-4a49-a5b5-72a4ce01b798-plugins-config-volume\") pod \"72a1a9df-dd32-4a49-a5b5-72a4ce01b798\" (UID: \"72a1a9df-dd32-4a49-a5b5-72a4ce01b798\") " Sep 08 18:53:45.017924 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:45.017898 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72a1a9df-dd32-4a49-a5b5-72a4ce01b798-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "72a1a9df-dd32-4a49-a5b5-72a4ce01b798" (UID: "72a1a9df-dd32-4a49-a5b5-72a4ce01b798"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:53:45.019777 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:45.019749 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72a1a9df-dd32-4a49-a5b5-72a4ce01b798-kube-api-access-tpdq6" (OuterVolumeSpecName: "kube-api-access-tpdq6") pod "72a1a9df-dd32-4a49-a5b5-72a4ce01b798" (UID: "72a1a9df-dd32-4a49-a5b5-72a4ce01b798"). InnerVolumeSpecName "kube-api-access-tpdq6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:53:45.118921 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:45.118897 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72a1a9df-dd32-4a49-a5b5-72a4ce01b798-plugins-config-volume\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:53:45.118921 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:45.118920 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tpdq6\" (UniqueName: \"kubernetes.io/projected/72a1a9df-dd32-4a49-a5b5-72a4ce01b798-kube-api-access-tpdq6\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:53:45.191657 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:45.191583 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:53:45.653263 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:45.653225 2555 generic.go:356] "Generic (PLEG): container finished" podID="72a1a9df-dd32-4a49-a5b5-72a4ce01b798" containerID="5737fe8def1fc2333a2cc65c112a0f7d51bf3ec31ef272fda43713c7a9b48f53" exitCode=0 Sep 08 18:53:45.653263 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:45.653264 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6255f2-5b689d4b8b-j58cg" Sep 08 18:53:45.653781 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:45.653285 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6255f2-5b689d4b8b-j58cg" event={"ID":"72a1a9df-dd32-4a49-a5b5-72a4ce01b798","Type":"ContainerDied","Data":"5737fe8def1fc2333a2cc65c112a0f7d51bf3ec31ef272fda43713c7a9b48f53"} Sep 08 18:53:45.653781 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:45.653338 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6255f2-5b689d4b8b-j58cg" event={"ID":"72a1a9df-dd32-4a49-a5b5-72a4ce01b798","Type":"ContainerDied","Data":"41aefbf89cc741c4b3e495e72922f2aa19f892e2dd236f634832d6d59de36525"} Sep 08 18:53:45.653781 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:45.653372 2555 scope.go:117] "RemoveContainer" containerID="5737fe8def1fc2333a2cc65c112a0f7d51bf3ec31ef272fda43713c7a9b48f53" Sep 08 18:53:45.663272 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:45.663100 2555 scope.go:117] "RemoveContainer" containerID="5737fe8def1fc2333a2cc65c112a0f7d51bf3ec31ef272fda43713c7a9b48f53" Sep 08 18:53:45.663390 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:53:45.663370 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5737fe8def1fc2333a2cc65c112a0f7d51bf3ec31ef272fda43713c7a9b48f53\": container with ID starting with 5737fe8def1fc2333a2cc65c112a0f7d51bf3ec31ef272fda43713c7a9b48f53 not found: ID does not exist" containerID="5737fe8def1fc2333a2cc65c112a0f7d51bf3ec31ef272fda43713c7a9b48f53" Sep 08 18:53:45.663432 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:45.663400 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5737fe8def1fc2333a2cc65c112a0f7d51bf3ec31ef272fda43713c7a9b48f53"} err="failed to get container status \"5737fe8def1fc2333a2cc65c112a0f7d51bf3ec31ef272fda43713c7a9b48f53\": rpc error: code = NotFound desc = could not find container \"5737fe8def1fc2333a2cc65c112a0f7d51bf3ec31ef272fda43713c7a9b48f53\": container with ID starting with 5737fe8def1fc2333a2cc65c112a0f7d51bf3ec31ef272fda43713c7a9b48f53 not found: ID does not exist" Sep 08 18:53:45.674907 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:45.674882 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6255f2-5b689d4b8b-j58cg"] Sep 08 18:53:45.679138 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:45.679115 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6255f2-5b689d4b8b-j58cg"] Sep 08 18:53:46.324738 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:46.324677 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33a34f27-be15-4794-9d9d-1250d3df9570" path="/var/lib/kubelet/pods/33a34f27-be15-4794-9d9d-1250d3df9570/volumes" Sep 08 18:53:46.326932 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:46.325280 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72a1a9df-dd32-4a49-a5b5-72a4ce01b798" path="/var/lib/kubelet/pods/72a1a9df-dd32-4a49-a5b5-72a4ce01b798/volumes" Sep 08 18:53:47.324434 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:47.324397 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:53:51.082717 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:51.082666 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:53:51.174708 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:51.174647 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:53:51.323482 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:51.323454 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt"] Sep 08 18:53:51.323730 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:51.323673 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" podUID="8ce8b8f9-b864-4072-bb13-92ad53c94917" containerName="istio-proxy" containerID="cri-o://7c0ebfb8c60356bd0820461dee0e2e1dfc4b664b2ccfc9b78898615cd6a51efa" gracePeriod=30 Sep 08 18:53:51.344042 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:51.343958 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:53:55.926073 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:55.926038 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:53:56.574821 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.574798 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:53:56.621914 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.621839 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ce8b8f9-b864-4072-bb13-92ad53c94917-istio-envoy\") pod \"8ce8b8f9-b864-4072-bb13-92ad53c94917\" (UID: \"8ce8b8f9-b864-4072-bb13-92ad53c94917\") " Sep 08 18:53:56.621914 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.621892 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hv2bw\" (UniqueName: \"kubernetes.io/projected/8ce8b8f9-b864-4072-bb13-92ad53c94917-kube-api-access-hv2bw\") pod \"8ce8b8f9-b864-4072-bb13-92ad53c94917\" (UID: \"8ce8b8f9-b864-4072-bb13-92ad53c94917\") " Sep 08 18:53:56.622149 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.621929 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ce8b8f9-b864-4072-bb13-92ad53c94917-workload-certs\") pod \"8ce8b8f9-b864-4072-bb13-92ad53c94917\" (UID: \"8ce8b8f9-b864-4072-bb13-92ad53c94917\") " Sep 08 18:53:56.622149 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.621948 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ce8b8f9-b864-4072-bb13-92ad53c94917-credential-socket\") pod \"8ce8b8f9-b864-4072-bb13-92ad53c94917\" (UID: \"8ce8b8f9-b864-4072-bb13-92ad53c94917\") " Sep 08 18:53:56.622149 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.621970 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ce8b8f9-b864-4072-bb13-92ad53c94917-istiod-ca-cert\") pod \"8ce8b8f9-b864-4072-bb13-92ad53c94917\" (UID: \"8ce8b8f9-b864-4072-bb13-92ad53c94917\") " Sep 08 18:53:56.622149 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.621988 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ce8b8f9-b864-4072-bb13-92ad53c94917-istio-data\") pod \"8ce8b8f9-b864-4072-bb13-92ad53c94917\" (UID: \"8ce8b8f9-b864-4072-bb13-92ad53c94917\") " Sep 08 18:53:56.622149 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.622020 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ce8b8f9-b864-4072-bb13-92ad53c94917-istio-podinfo\") pod \"8ce8b8f9-b864-4072-bb13-92ad53c94917\" (UID: \"8ce8b8f9-b864-4072-bb13-92ad53c94917\") " Sep 08 18:53:56.622149 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.622076 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ce8b8f9-b864-4072-bb13-92ad53c94917-istio-token\") pod \"8ce8b8f9-b864-4072-bb13-92ad53c94917\" (UID: \"8ce8b8f9-b864-4072-bb13-92ad53c94917\") " Sep 08 18:53:56.622149 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.622101 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ce8b8f9-b864-4072-bb13-92ad53c94917-workload-socket\") pod \"8ce8b8f9-b864-4072-bb13-92ad53c94917\" (UID: \"8ce8b8f9-b864-4072-bb13-92ad53c94917\") " Sep 08 18:53:56.622513 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.622265 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ce8b8f9-b864-4072-bb13-92ad53c94917-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8ce8b8f9-b864-4072-bb13-92ad53c94917" (UID: "8ce8b8f9-b864-4072-bb13-92ad53c94917"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:53:56.622513 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.622369 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ce8b8f9-b864-4072-bb13-92ad53c94917-istio-data" (OuterVolumeSpecName: "istio-data") pod "8ce8b8f9-b864-4072-bb13-92ad53c94917" (UID: "8ce8b8f9-b864-4072-bb13-92ad53c94917"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:53:56.622513 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.622382 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ce8b8f9-b864-4072-bb13-92ad53c94917-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8ce8b8f9-b864-4072-bb13-92ad53c94917" (UID: "8ce8b8f9-b864-4072-bb13-92ad53c94917"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:53:56.622513 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.622490 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ce8b8f9-b864-4072-bb13-92ad53c94917-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8ce8b8f9-b864-4072-bb13-92ad53c94917" (UID: "8ce8b8f9-b864-4072-bb13-92ad53c94917"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:53:56.622810 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.622564 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ce8b8f9-b864-4072-bb13-92ad53c94917-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8ce8b8f9-b864-4072-bb13-92ad53c94917" (UID: "8ce8b8f9-b864-4072-bb13-92ad53c94917"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 08 18:53:56.622810 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.622633 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ce8b8f9-b864-4072-bb13-92ad53c94917-credential-socket\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:53:56.622810 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.622651 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ce8b8f9-b864-4072-bb13-92ad53c94917-istio-data\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:53:56.622810 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.622660 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ce8b8f9-b864-4072-bb13-92ad53c94917-workload-socket\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:53:56.622810 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.622671 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ce8b8f9-b864-4072-bb13-92ad53c94917-workload-certs\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:53:56.624174 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.624148 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ce8b8f9-b864-4072-bb13-92ad53c94917-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8ce8b8f9-b864-4072-bb13-92ad53c94917" (UID: "8ce8b8f9-b864-4072-bb13-92ad53c94917"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 08 18:53:56.624376 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.624352 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8ce8b8f9-b864-4072-bb13-92ad53c94917-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8ce8b8f9-b864-4072-bb13-92ad53c94917" (UID: "8ce8b8f9-b864-4072-bb13-92ad53c94917"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 08 18:53:56.624500 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.624482 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ce8b8f9-b864-4072-bb13-92ad53c94917-istio-token" (OuterVolumeSpecName: "istio-token") pod "8ce8b8f9-b864-4072-bb13-92ad53c94917" (UID: "8ce8b8f9-b864-4072-bb13-92ad53c94917"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:53:56.624547 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.624496 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ce8b8f9-b864-4072-bb13-92ad53c94917-kube-api-access-hv2bw" (OuterVolumeSpecName: "kube-api-access-hv2bw") pod "8ce8b8f9-b864-4072-bb13-92ad53c94917" (UID: "8ce8b8f9-b864-4072-bb13-92ad53c94917"). InnerVolumeSpecName "kube-api-access-hv2bw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 08 18:53:56.714330 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.714297 2555 generic.go:356] "Generic (PLEG): container finished" podID="8ce8b8f9-b864-4072-bb13-92ad53c94917" containerID="7c0ebfb8c60356bd0820461dee0e2e1dfc4b664b2ccfc9b78898615cd6a51efa" exitCode=0 Sep 08 18:53:56.714502 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.714340 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" Sep 08 18:53:56.714502 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.714375 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" event={"ID":"8ce8b8f9-b864-4072-bb13-92ad53c94917","Type":"ContainerDied","Data":"7c0ebfb8c60356bd0820461dee0e2e1dfc4b664b2ccfc9b78898615cd6a51efa"} Sep 08 18:53:56.714502 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.714423 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt" event={"ID":"8ce8b8f9-b864-4072-bb13-92ad53c94917","Type":"ContainerDied","Data":"31c9cbe142de28541037e4e4b0d35a1bbd37f725701a3fa693b36cccd2a984fb"} Sep 08 18:53:56.714502 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.714446 2555 scope.go:117] "RemoveContainer" containerID="7c0ebfb8c60356bd0820461dee0e2e1dfc4b664b2ccfc9b78898615cd6a51efa" Sep 08 18:53:56.723623 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.723603 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ce8b8f9-b864-4072-bb13-92ad53c94917-istio-envoy\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:53:56.723623 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.723624 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hv2bw\" (UniqueName: \"kubernetes.io/projected/8ce8b8f9-b864-4072-bb13-92ad53c94917-kube-api-access-hv2bw\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:53:56.723791 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.723634 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ce8b8f9-b864-4072-bb13-92ad53c94917-istiod-ca-cert\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:53:56.723791 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.723644 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ce8b8f9-b864-4072-bb13-92ad53c94917-istio-podinfo\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:53:56.723791 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.723652 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ce8b8f9-b864-4072-bb13-92ad53c94917-istio-token\") on node \"ip-10-0-139-112.ec2.internal\" DevicePath \"\"" Sep 08 18:53:56.724464 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.724453 2555 scope.go:117] "RemoveContainer" containerID="56aeee8f913e923028600b7de31543839079e013ec480ddcabdd338d9cf23353" Sep 08 18:53:56.733166 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.733146 2555 scope.go:117] "RemoveContainer" containerID="7c0ebfb8c60356bd0820461dee0e2e1dfc4b664b2ccfc9b78898615cd6a51efa" Sep 08 18:53:56.733500 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:53:56.733480 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c0ebfb8c60356bd0820461dee0e2e1dfc4b664b2ccfc9b78898615cd6a51efa\": container with ID starting with 7c0ebfb8c60356bd0820461dee0e2e1dfc4b664b2ccfc9b78898615cd6a51efa not found: ID does not exist" containerID="7c0ebfb8c60356bd0820461dee0e2e1dfc4b664b2ccfc9b78898615cd6a51efa" Sep 08 18:53:56.733599 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.733513 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c0ebfb8c60356bd0820461dee0e2e1dfc4b664b2ccfc9b78898615cd6a51efa"} err="failed to get container status \"7c0ebfb8c60356bd0820461dee0e2e1dfc4b664b2ccfc9b78898615cd6a51efa\": rpc error: code = NotFound desc = could not find container \"7c0ebfb8c60356bd0820461dee0e2e1dfc4b664b2ccfc9b78898615cd6a51efa\": container with ID starting with 7c0ebfb8c60356bd0820461dee0e2e1dfc4b664b2ccfc9b78898615cd6a51efa not found: ID does not exist" Sep 08 18:53:56.733599 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.733545 2555 scope.go:117] "RemoveContainer" containerID="56aeee8f913e923028600b7de31543839079e013ec480ddcabdd338d9cf23353" Sep 08 18:53:56.733984 ip-10-0-139-112 kubenswrapper[2555]: E0908 18:53:56.733959 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56aeee8f913e923028600b7de31543839079e013ec480ddcabdd338d9cf23353\": container with ID starting with 56aeee8f913e923028600b7de31543839079e013ec480ddcabdd338d9cf23353 not found: ID does not exist" containerID="56aeee8f913e923028600b7de31543839079e013ec480ddcabdd338d9cf23353" Sep 08 18:53:56.734051 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.733995 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56aeee8f913e923028600b7de31543839079e013ec480ddcabdd338d9cf23353"} err="failed to get container status \"56aeee8f913e923028600b7de31543839079e013ec480ddcabdd338d9cf23353\": rpc error: code = NotFound desc = could not find container \"56aeee8f913e923028600b7de31543839079e013ec480ddcabdd338d9cf23353\": container with ID starting with 56aeee8f913e923028600b7de31543839079e013ec480ddcabdd338d9cf23353 not found: ID does not exist" Sep 08 18:53:56.735548 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.735531 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt"] Sep 08 18:53:56.741295 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:56.741276 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-6255f2-openshift-default-5854ff94c4-2s2lt"] Sep 08 18:53:58.324387 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:53:58.324352 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ce8b8f9-b864-4072-bb13-92ad53c94917" path="/var/lib/kubelet/pods/8ce8b8f9-b864-4072-bb13-92ad53c94917/volumes" Sep 08 18:54:14.065528 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:54:14.065476 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:54:14.145890 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:54:14.145862 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:54:38.914769 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:54:38.914736 2555 scope.go:117] "RemoveContainer" containerID="ac6d5f3ed94fb3fee2a57ee7003cad1c75dfc54ce53959608ad6244527bb6ae0" Sep 08 18:54:38.923883 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:54:38.923863 2555 scope.go:117] "RemoveContainer" containerID="03737ea2c427e69c29f1aeb1e1f53c2ce7000cb7b77ff59b21ca19711b05c51d" Sep 08 18:55:15.465819 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:55:15.465775 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:55:16.625376 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:55:16.625335 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:55:17.424889 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:55:17.424840 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:55:18.943797 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:55:18.943743 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:55:43.626308 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:55:43.626268 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:55:51.220215 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:55:51.220181 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:55:58.417892 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:55:58.417854 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:56:06.022012 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:56:06.021971 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:56:19.828643 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:56:19.828601 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:56:27.730328 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:56:27.730293 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:57:45.368232 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:57:45.368190 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:57:54.640568 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:57:54.640528 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:57:54.724291 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:57:54.724253 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:58:01.230801 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:58:01.230713 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:58:08.140089 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:58:08.140046 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:58:16.052473 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:58:16.052433 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:58:18.634127 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:58:18.634091 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:58:32.299873 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:58:32.299831 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:58:35.443568 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:58:35.443531 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:58:38.517975 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:58:38.517948 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9n28f_911c61b8-766b-4ae8-b046-784e22e85889/console-operator/1.log" Sep 08 18:58:38.527157 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:58:38.527136 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9n28f_911c61b8-766b-4ae8-b046-784e22e85889/console-operator/1.log" Sep 08 18:59:09.693013 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:09.692977 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:59:11.030295 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:11.030256 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:59:11.732718 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:11.732670 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:59:19.987438 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:19.987398 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 18:59:20.324540 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:20.324512 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:59:26.829186 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:26.829148 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:59:34.524119 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:34.524042 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-79cd597bc5-6n4rx"] Sep 08 18:59:34.524552 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:34.524437 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed5d375f-bf6f-4657-b9d3-5be0d6874942" containerName="revoke-keys" Sep 08 18:59:34.524552 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:34.524448 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed5d375f-bf6f-4657-b9d3-5be0d6874942" containerName="revoke-keys" Sep 08 18:59:34.524552 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:34.524457 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72a1a9df-dd32-4a49-a5b5-72a4ce01b798" containerName="payload-pre-processing" Sep 08 18:59:34.524552 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:34.524464 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="72a1a9df-dd32-4a49-a5b5-72a4ce01b798" containerName="payload-pre-processing" Sep 08 18:59:34.524552 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:34.524481 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ce8b8f9-b864-4072-bb13-92ad53c94917" containerName="istio-proxy" Sep 08 18:59:34.524552 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:34.524487 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ce8b8f9-b864-4072-bb13-92ad53c94917" containerName="istio-proxy" Sep 08 18:59:34.524552 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:34.524555 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ce8b8f9-b864-4072-bb13-92ad53c94917" containerName="istio-proxy" Sep 08 18:59:34.524808 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:34.524562 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="72a1a9df-dd32-4a49-a5b5-72a4ce01b798" containerName="payload-pre-processing" Sep 08 18:59:34.524808 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:34.524571 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ce8b8f9-b864-4072-bb13-92ad53c94917" containerName="istio-proxy" Sep 08 18:59:34.524808 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:34.524577 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed5d375f-bf6f-4657-b9d3-5be0d6874942" containerName="revoke-keys" Sep 08 18:59:34.527788 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:34.527773 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-79cd597bc5-6n4rx" Sep 08 18:59:34.530447 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:34.530422 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Sep 08 18:59:34.530596 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:34.530454 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Sep 08 18:59:34.530596 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:34.530425 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-wvhnh\"" Sep 08 18:59:34.530596 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:34.530427 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Sep 08 18:59:34.530596 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:34.530427 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 08 18:59:34.538066 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:34.538042 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-79cd597bc5-6n4rx"] Sep 08 18:59:34.639496 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:34.639454 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w62b8\" (UniqueName: \"kubernetes.io/projected/42f764f9-87cb-4c8f-8de7-dd103a4e7b2e-kube-api-access-w62b8\") pod \"maas-api-79cd597bc5-6n4rx\" (UID: \"42f764f9-87cb-4c8f-8de7-dd103a4e7b2e\") " pod="odh-ai-gateway-infra/maas-api-79cd597bc5-6n4rx" Sep 08 18:59:34.639496 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:34.639491 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42f764f9-87cb-4c8f-8de7-dd103a4e7b2e-metrics-certs\") pod \"maas-api-79cd597bc5-6n4rx\" (UID: \"42f764f9-87cb-4c8f-8de7-dd103a4e7b2e\") " pod="odh-ai-gateway-infra/maas-api-79cd597bc5-6n4rx" Sep 08 18:59:34.639844 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:34.639589 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42f764f9-87cb-4c8f-8de7-dd103a4e7b2e-openshift-service-ca\") pod \"maas-api-79cd597bc5-6n4rx\" (UID: \"42f764f9-87cb-4c8f-8de7-dd103a4e7b2e\") " pod="odh-ai-gateway-infra/maas-api-79cd597bc5-6n4rx" Sep 08 18:59:34.639844 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:34.639752 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42f764f9-87cb-4c8f-8de7-dd103a4e7b2e-maas-api-tls\") pod \"maas-api-79cd597bc5-6n4rx\" (UID: \"42f764f9-87cb-4c8f-8de7-dd103a4e7b2e\") " pod="odh-ai-gateway-infra/maas-api-79cd597bc5-6n4rx" Sep 08 18:59:34.727927 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:34.727892 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:59:34.740654 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:34.740623 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w62b8\" (UniqueName: \"kubernetes.io/projected/42f764f9-87cb-4c8f-8de7-dd103a4e7b2e-kube-api-access-w62b8\") pod \"maas-api-79cd597bc5-6n4rx\" (UID: \"42f764f9-87cb-4c8f-8de7-dd103a4e7b2e\") " pod="odh-ai-gateway-infra/maas-api-79cd597bc5-6n4rx" Sep 08 18:59:34.740654 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:34.740655 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42f764f9-87cb-4c8f-8de7-dd103a4e7b2e-metrics-certs\") pod \"maas-api-79cd597bc5-6n4rx\" (UID: \"42f764f9-87cb-4c8f-8de7-dd103a4e7b2e\") " pod="odh-ai-gateway-infra/maas-api-79cd597bc5-6n4rx" Sep 08 18:59:34.740877 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:34.740858 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42f764f9-87cb-4c8f-8de7-dd103a4e7b2e-openshift-service-ca\") pod \"maas-api-79cd597bc5-6n4rx\" (UID: \"42f764f9-87cb-4c8f-8de7-dd103a4e7b2e\") " pod="odh-ai-gateway-infra/maas-api-79cd597bc5-6n4rx" Sep 08 18:59:34.740953 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:34.740934 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42f764f9-87cb-4c8f-8de7-dd103a4e7b2e-maas-api-tls\") pod \"maas-api-79cd597bc5-6n4rx\" (UID: \"42f764f9-87cb-4c8f-8de7-dd103a4e7b2e\") " pod="odh-ai-gateway-infra/maas-api-79cd597bc5-6n4rx" Sep 08 18:59:34.741473 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:34.741448 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42f764f9-87cb-4c8f-8de7-dd103a4e7b2e-openshift-service-ca\") pod \"maas-api-79cd597bc5-6n4rx\" (UID: \"42f764f9-87cb-4c8f-8de7-dd103a4e7b2e\") " pod="odh-ai-gateway-infra/maas-api-79cd597bc5-6n4rx" Sep 08 18:59:34.743283 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:34.743254 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42f764f9-87cb-4c8f-8de7-dd103a4e7b2e-metrics-certs\") pod \"maas-api-79cd597bc5-6n4rx\" (UID: \"42f764f9-87cb-4c8f-8de7-dd103a4e7b2e\") " pod="odh-ai-gateway-infra/maas-api-79cd597bc5-6n4rx" Sep 08 18:59:34.743283 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:34.743277 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42f764f9-87cb-4c8f-8de7-dd103a4e7b2e-maas-api-tls\") pod \"maas-api-79cd597bc5-6n4rx\" (UID: \"42f764f9-87cb-4c8f-8de7-dd103a4e7b2e\") " pod="odh-ai-gateway-infra/maas-api-79cd597bc5-6n4rx" Sep 08 18:59:34.748509 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:34.748485 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w62b8\" (UniqueName: \"kubernetes.io/projected/42f764f9-87cb-4c8f-8de7-dd103a4e7b2e-kube-api-access-w62b8\") pod \"maas-api-79cd597bc5-6n4rx\" (UID: \"42f764f9-87cb-4c8f-8de7-dd103a4e7b2e\") " pod="odh-ai-gateway-infra/maas-api-79cd597bc5-6n4rx" Sep 08 18:59:34.839135 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:34.839097 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-79cd597bc5-6n4rx" Sep 08 18:59:35.114523 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:35.114495 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-79cd597bc5-6n4rx"] Sep 08 18:59:35.116398 ip-10-0-139-112 kubenswrapper[2555]: W0908 18:59:35.116368 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42f764f9_87cb_4c8f_8de7_dd103a4e7b2e.slice/crio-9e898844bfe7787573d20fc2a3dd1184846ed410f738246e1e1d658c84a3211b WatchSource:0}: Error finding container 9e898844bfe7787573d20fc2a3dd1184846ed410f738246e1e1d658c84a3211b: Status 404 returned error can't find the container with id 9e898844bfe7787573d20fc2a3dd1184846ed410f738246e1e1d658c84a3211b Sep 08 18:59:35.117758 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:35.117737 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 08 18:59:35.622459 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:35.622420 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-79cd597bc5-6n4rx" event={"ID":"42f764f9-87cb-4c8f-8de7-dd103a4e7b2e","Type":"ContainerStarted","Data":"9e898844bfe7787573d20fc2a3dd1184846ed410f738246e1e1d658c84a3211b"} Sep 08 18:59:37.635623 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:37.635584 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-79cd597bc5-6n4rx" event={"ID":"42f764f9-87cb-4c8f-8de7-dd103a4e7b2e","Type":"ContainerStarted","Data":"5a7eed223f63145ec907a464ea0649a44641bb98405e624f6a2e6970b3a6b206"} Sep 08 18:59:37.636046 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:37.635750 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-79cd597bc5-6n4rx" Sep 08 18:59:37.654536 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:37.654491 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-79cd597bc5-6n4rx" podStartSLOduration=1.981890814 podStartE2EDuration="3.654477527s" totalImagesPullingTime="1.672586713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 18:59:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 18:59:37.652102746 +0000 UTC m=+2159.858088142" watchObservedRunningTime="2026-09-08 18:59:37.654477527 +0000 UTC m=+2159.860462923" Sep 08 18:59:38.994395 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:38.994283 2555 scope.go:117] "RemoveContainer" containerID="82a44a81f6d144dd704b5f5f73ef7ab6dde81c3c147b54b4a56d580365f9f3c3" Sep 08 18:59:41.531118 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:41.531072 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:59:43.646261 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:43.646231 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-79cd597bc5-6n4rx" Sep 08 18:59:47.127904 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:47.127860 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:59:53.822739 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:53.822699 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-prfq4"] Sep 08 18:59:56.886386 ip-10-0-139-112 kubenswrapper[2555]: I0908 18:59:56.886358 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6d7947bbf4-z8l5z_c4f2cdc0-591a-4a3e-8ad1-d4010e3203de/authorino/0.log" Sep 08 19:00:03.094496 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:03.094466 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8d688d88f-l42d7_684038e9-9cd1-457c-b852-d6eabc24e260/copy-manifests/0.log" Sep 08 19:00:03.102705 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:03.102662 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8d688d88f-l42d7_684038e9-9cd1-457c-b852-d6eabc24e260/manager/0.log" Sep 08 19:00:03.454897 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:03.454812 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-79cd597bc5-6n4rx_42f764f9-87cb-4c8f-8de7-dd103a4e7b2e/maas-api/0.log" Sep 08 19:00:04.723503 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:04.723478 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6d7947bbf4-z8l5z_c4f2cdc0-591a-4a3e-8ad1-d4010e3203de/authorino/0.log" Sep 08 19:00:05.085968 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:05.085938 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-xqjvm_fe663dac-2e58-406f-92f2-9ea9868b662f/kuadrant-console-plugin/0.log" Sep 08 19:00:05.334169 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:05.334141 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-prfq4_b033b298-b4fe-4dbc-bf2e-119bd2723454/limitador/0.log" Sep 08 19:00:06.171858 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:06.171826 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6d985cc74d-b2kg9_7c1b6bd1-08a0-4fce-be8c-648216b13ebe/kube-auth-proxy/0.log" Sep 08 19:00:06.650131 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:06.650101 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7bdcf569fd-2mtjd_c44b4985-58c8-4e47-af99-0fd9052f8c7b/router/0.log" Sep 08 19:00:07.234154 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:07.234118 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-dxlnw_dd2423f3-b698-4319-84bd-34fc45ce7e7b/main/0.log" Sep 08 19:00:07.694921 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:07.694886 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-tv6zd_ca6972fc-daa1-4d54-a149-ab81b4187e5c/main/0.log" Sep 08 19:00:13.799938 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:13.799900 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 19:00:13.871025 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:13.870992 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" limit=1024 Sep 08 19:00:16.845417 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:16.845387 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-qbksq_2744e789-a69b-4c63-b2db-2a7676c746ca/global-pull-secret-syncer/0.log" Sep 08 19:00:16.951340 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:16.951256 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-29snt_cfadca84-d487-4e8d-8c7c-12369bab945d/konnectivity-agent/0.log" Sep 08 19:00:17.056543 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:17.056516 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-112.ec2.internal_c532783d472be20cfe3b62023aa174a5/haproxy/0.log" Sep 08 19:00:21.949026 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:21.948990 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6d7947bbf4-z8l5z_c4f2cdc0-591a-4a3e-8ad1-d4010e3203de/authorino/0.log" Sep 08 19:00:22.041664 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:22.041638 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-xqjvm_fe663dac-2e58-406f-92f2-9ea9868b662f/kuadrant-console-plugin/0.log" Sep 08 19:00:22.157808 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:22.157775 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-prfq4_b033b298-b4fe-4dbc-bf2e-119bd2723454/limitador/0.log" Sep 08 19:00:23.614671 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:23.614641 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ee96a90f-f261-4b6e-a081-0342c099e5a0/alertmanager/0.log" Sep 08 19:00:23.638564 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:23.638536 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ee96a90f-f261-4b6e-a081-0342c099e5a0/config-reloader/0.log" Sep 08 19:00:23.664681 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:23.664654 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ee96a90f-f261-4b6e-a081-0342c099e5a0/kube-rbac-proxy-web/0.log" Sep 08 19:00:23.687965 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:23.687935 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ee96a90f-f261-4b6e-a081-0342c099e5a0/kube-rbac-proxy/0.log" Sep 08 19:00:23.710025 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:23.709991 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ee96a90f-f261-4b6e-a081-0342c099e5a0/kube-rbac-proxy-metric/0.log" Sep 08 19:00:23.736509 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:23.736483 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ee96a90f-f261-4b6e-a081-0342c099e5a0/prom-label-proxy/0.log" Sep 08 19:00:23.759609 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:23.759583 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ee96a90f-f261-4b6e-a081-0342c099e5a0/init-config-reloader/0.log" Sep 08 19:00:23.951480 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:23.951387 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-ctvlx_b91f2cbc-e834-418d-a40b-e2387fac9385/monitoring-plugin/0.log" Sep 08 19:00:24.185009 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:24.184983 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mdqr6_7e607d22-ab60-4fcc-88ad-ca0acfcf3642/node-exporter/0.log" Sep 08 19:00:24.210184 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:24.210100 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mdqr6_7e607d22-ab60-4fcc-88ad-ca0acfcf3642/kube-rbac-proxy/0.log" Sep 08 19:00:24.239844 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:24.239816 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mdqr6_7e607d22-ab60-4fcc-88ad-ca0acfcf3642/init-textfile/0.log" Sep 08 19:00:24.272377 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:24.272350 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-dpnvv_06e53c2f-304e-403c-9ed9-d107bbcdf01c/kube-rbac-proxy-main/0.log" Sep 08 19:00:24.304160 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:24.304122 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-dpnvv_06e53c2f-304e-403c-9ed9-d107bbcdf01c/kube-rbac-proxy-self/0.log" Sep 08 19:00:24.331056 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:24.331024 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-dpnvv_06e53c2f-304e-403c-9ed9-d107bbcdf01c/openshift-state-metrics/0.log" Sep 08 19:00:24.593220 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:24.593193 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-ggzxh_1235e212-ee01-4797-8566-fc2b904316fd/prometheus-operator/0.log" Sep 08 19:00:24.616804 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:24.616779 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-ggzxh_1235e212-ee01-4797-8566-fc2b904316fd/kube-rbac-proxy/0.log" Sep 08 19:00:24.690574 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:24.690525 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6d85bb5d46-bxjxt_96fcab03-2331-4636-881c-eb6292fe6243/telemeter-client/0.log" Sep 08 19:00:24.716156 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:24.716126 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6d85bb5d46-bxjxt_96fcab03-2331-4636-881c-eb6292fe6243/reload/0.log" Sep 08 19:00:24.744830 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:24.744800 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6d85bb5d46-bxjxt_96fcab03-2331-4636-881c-eb6292fe6243/kube-rbac-proxy/0.log" Sep 08 19:00:24.779024 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:24.778995 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-547cd64fc8-vx522_c5505916-4939-4ed0-8168-a469bcb150a9/thanos-query/0.log" Sep 08 19:00:24.803507 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:24.803483 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-547cd64fc8-vx522_c5505916-4939-4ed0-8168-a469bcb150a9/kube-rbac-proxy-web/0.log" Sep 08 19:00:24.827010 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:24.826973 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-547cd64fc8-vx522_c5505916-4939-4ed0-8168-a469bcb150a9/kube-rbac-proxy/0.log" Sep 08 19:00:24.854203 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:24.854119 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-547cd64fc8-vx522_c5505916-4939-4ed0-8168-a469bcb150a9/prom-label-proxy/0.log" Sep 08 19:00:24.876844 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:24.876814 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-547cd64fc8-vx522_c5505916-4939-4ed0-8168-a469bcb150a9/kube-rbac-proxy-rules/0.log" Sep 08 19:00:24.898585 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:24.898527 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-547cd64fc8-vx522_c5505916-4939-4ed0-8168-a469bcb150a9/kube-rbac-proxy-metrics/0.log" Sep 08 19:00:26.788626 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:26.788589 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9n28f_911c61b8-766b-4ae8-b046-784e22e85889/console-operator/1.log" Sep 08 19:00:26.799510 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:26.799488 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9n28f_911c61b8-766b-4ae8-b046-784e22e85889/console-operator/2.log" Sep 08 19:00:27.034546 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:27.034508 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-phw82/perf-node-gather-daemonset-86nrx"] Sep 08 19:00:27.034935 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:27.034921 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ce8b8f9-b864-4072-bb13-92ad53c94917" containerName="istio-proxy" Sep 08 19:00:27.034994 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:27.034937 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ce8b8f9-b864-4072-bb13-92ad53c94917" containerName="istio-proxy" Sep 08 19:00:27.038197 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:27.038181 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-phw82/perf-node-gather-daemonset-86nrx" Sep 08 19:00:27.040301 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:27.040226 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-phw82\"/\"kube-root-ca.crt\"" Sep 08 19:00:27.040844 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:27.040828 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-phw82\"/\"default-dockercfg-fm6tw\"" Sep 08 19:00:27.040905 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:27.040830 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-phw82\"/\"openshift-service-ca.crt\"" Sep 08 19:00:27.046361 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:27.046228 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-phw82/perf-node-gather-daemonset-86nrx"] Sep 08 19:00:27.123027 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:27.122983 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f83aef7c-542a-4db6-810f-3cfa0936e8f7-proc\") pod \"perf-node-gather-daemonset-86nrx\" (UID: \"f83aef7c-542a-4db6-810f-3cfa0936e8f7\") " pod="openshift-must-gather-phw82/perf-node-gather-daemonset-86nrx" Sep 08 19:00:27.123253 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:27.123048 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f83aef7c-542a-4db6-810f-3cfa0936e8f7-sys\") pod \"perf-node-gather-daemonset-86nrx\" (UID: \"f83aef7c-542a-4db6-810f-3cfa0936e8f7\") " pod="openshift-must-gather-phw82/perf-node-gather-daemonset-86nrx" Sep 08 19:00:27.123253 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:27.123157 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f83aef7c-542a-4db6-810f-3cfa0936e8f7-podres\") pod \"perf-node-gather-daemonset-86nrx\" (UID: \"f83aef7c-542a-4db6-810f-3cfa0936e8f7\") " pod="openshift-must-gather-phw82/perf-node-gather-daemonset-86nrx" Sep 08 19:00:27.123253 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:27.123191 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbgx7\" (UniqueName: \"kubernetes.io/projected/f83aef7c-542a-4db6-810f-3cfa0936e8f7-kube-api-access-sbgx7\") pod \"perf-node-gather-daemonset-86nrx\" (UID: \"f83aef7c-542a-4db6-810f-3cfa0936e8f7\") " pod="openshift-must-gather-phw82/perf-node-gather-daemonset-86nrx" Sep 08 19:00:27.123383 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:27.123248 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f83aef7c-542a-4db6-810f-3cfa0936e8f7-lib-modules\") pod \"perf-node-gather-daemonset-86nrx\" (UID: \"f83aef7c-542a-4db6-810f-3cfa0936e8f7\") " pod="openshift-must-gather-phw82/perf-node-gather-daemonset-86nrx" Sep 08 19:00:27.224625 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:27.224585 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f83aef7c-542a-4db6-810f-3cfa0936e8f7-sys\") pod \"perf-node-gather-daemonset-86nrx\" (UID: \"f83aef7c-542a-4db6-810f-3cfa0936e8f7\") " pod="openshift-must-gather-phw82/perf-node-gather-daemonset-86nrx" Sep 08 19:00:27.224906 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:27.224642 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f83aef7c-542a-4db6-810f-3cfa0936e8f7-podres\") pod \"perf-node-gather-daemonset-86nrx\" (UID: \"f83aef7c-542a-4db6-810f-3cfa0936e8f7\") " pod="openshift-must-gather-phw82/perf-node-gather-daemonset-86nrx" Sep 08 19:00:27.224906 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:27.224659 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbgx7\" (UniqueName: \"kubernetes.io/projected/f83aef7c-542a-4db6-810f-3cfa0936e8f7-kube-api-access-sbgx7\") pod \"perf-node-gather-daemonset-86nrx\" (UID: \"f83aef7c-542a-4db6-810f-3cfa0936e8f7\") " pod="openshift-must-gather-phw82/perf-node-gather-daemonset-86nrx" Sep 08 19:00:27.224906 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:27.224743 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f83aef7c-542a-4db6-810f-3cfa0936e8f7-sys\") pod \"perf-node-gather-daemonset-86nrx\" (UID: \"f83aef7c-542a-4db6-810f-3cfa0936e8f7\") " pod="openshift-must-gather-phw82/perf-node-gather-daemonset-86nrx" Sep 08 19:00:27.224906 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:27.224803 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f83aef7c-542a-4db6-810f-3cfa0936e8f7-lib-modules\") pod \"perf-node-gather-daemonset-86nrx\" (UID: \"f83aef7c-542a-4db6-810f-3cfa0936e8f7\") " pod="openshift-must-gather-phw82/perf-node-gather-daemonset-86nrx" Sep 08 19:00:27.224906 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:27.224872 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f83aef7c-542a-4db6-810f-3cfa0936e8f7-proc\") pod \"perf-node-gather-daemonset-86nrx\" (UID: \"f83aef7c-542a-4db6-810f-3cfa0936e8f7\") " pod="openshift-must-gather-phw82/perf-node-gather-daemonset-86nrx" Sep 08 19:00:27.224906 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:27.224895 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f83aef7c-542a-4db6-810f-3cfa0936e8f7-lib-modules\") pod \"perf-node-gather-daemonset-86nrx\" (UID: \"f83aef7c-542a-4db6-810f-3cfa0936e8f7\") " pod="openshift-must-gather-phw82/perf-node-gather-daemonset-86nrx" Sep 08 19:00:27.225154 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:27.224805 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f83aef7c-542a-4db6-810f-3cfa0936e8f7-podres\") pod \"perf-node-gather-daemonset-86nrx\" (UID: \"f83aef7c-542a-4db6-810f-3cfa0936e8f7\") " pod="openshift-must-gather-phw82/perf-node-gather-daemonset-86nrx" Sep 08 19:00:27.225154 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:27.224974 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f83aef7c-542a-4db6-810f-3cfa0936e8f7-proc\") pod \"perf-node-gather-daemonset-86nrx\" (UID: \"f83aef7c-542a-4db6-810f-3cfa0936e8f7\") " pod="openshift-must-gather-phw82/perf-node-gather-daemonset-86nrx" Sep 08 19:00:27.232152 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:27.232128 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbgx7\" (UniqueName: \"kubernetes.io/projected/f83aef7c-542a-4db6-810f-3cfa0936e8f7-kube-api-access-sbgx7\") pod \"perf-node-gather-daemonset-86nrx\" (UID: \"f83aef7c-542a-4db6-810f-3cfa0936e8f7\") " pod="openshift-must-gather-phw82/perf-node-gather-daemonset-86nrx" Sep 08 19:00:27.349844 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:27.349743 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-phw82/perf-node-gather-daemonset-86nrx" Sep 08 19:00:27.391330 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:27.391292 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7cbbd74665-bpntq_117aa701-7af8-4fc8-9477-74e60ae6e463/console/0.log" Sep 08 19:00:27.440218 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:27.440186 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-ctmck_b514b325-9daf-4c8f-a8c1-cee65d51c2f0/download-server/0.log" Sep 08 19:00:27.631811 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:27.631717 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-phw82/perf-node-gather-daemonset-86nrx"] Sep 08 19:00:27.635419 ip-10-0-139-112 kubenswrapper[2555]: W0908 19:00:27.635393 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf83aef7c_542a_4db6_810f_3cfa0936e8f7.slice/crio-9ba452033ab94a7e03181f9b9f3ec6b3daa15ae0cc8860b8213879fcf9faa520 WatchSource:0}: Error finding container 9ba452033ab94a7e03181f9b9f3ec6b3daa15ae0cc8860b8213879fcf9faa520: Status 404 returned error can't find the container with id 9ba452033ab94a7e03181f9b9f3ec6b3daa15ae0cc8860b8213879fcf9faa520 Sep 08 19:00:27.935173 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:27.935082 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-phw82/perf-node-gather-daemonset-86nrx" event={"ID":"f83aef7c-542a-4db6-810f-3cfa0936e8f7","Type":"ContainerStarted","Data":"d8f68ea013246e4e8fe07391b1f0400ac72d26d4488385143346b92911eafc5a"} Sep 08 19:00:27.935173 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:27.935136 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-phw82/perf-node-gather-daemonset-86nrx" event={"ID":"f83aef7c-542a-4db6-810f-3cfa0936e8f7","Type":"ContainerStarted","Data":"9ba452033ab94a7e03181f9b9f3ec6b3daa15ae0cc8860b8213879fcf9faa520"} Sep 08 19:00:27.935173 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:27.935160 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-phw82/perf-node-gather-daemonset-86nrx" Sep 08 19:00:27.951455 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:27.951401 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-phw82/perf-node-gather-daemonset-86nrx" podStartSLOduration=0.951384482 podStartE2EDuration="951.384482ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 19:00:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 19:00:27.949625396 +0000 UTC m=+2210.155610803" watchObservedRunningTime="2026-09-08 19:00:27.951384482 +0000 UTC m=+2210.157369880" Sep 08 19:00:29.337309 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:29.337280 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pzzmm_9de9b23a-f1e9-4b44-87a2-680997e061f0/dns/0.log" Sep 08 19:00:29.362381 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:29.362355 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pzzmm_9de9b23a-f1e9-4b44-87a2-680997e061f0/kube-rbac-proxy/0.log" Sep 08 19:00:29.416239 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:29.416206 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lnhcj_582b7b23-efed-41a0-a58f-726efcdff3a8/dns-node-resolver/0.log" Sep 08 19:00:29.990023 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:29.989998 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dwsrv_e5d99ca2-78e9-43bb-946d-a944ffd92d62/node-ca/0.log" Sep 08 19:00:31.483151 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:31.483119 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6d985cc74d-b2kg9_7c1b6bd1-08a0-4fce-be8c-648216b13ebe/kube-auth-proxy/0.log" Sep 08 19:00:31.637479 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:31.637434 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7bdcf569fd-2mtjd_c44b4985-58c8-4e47-af99-0fd9052f8c7b/router/0.log" Sep 08 19:00:32.556009 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:32.555981 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-k4dxz_a067ecec-0c84-4299-812c-6441d126104f/serve-healthcheck-canary/0.log" Sep 08 19:00:33.067095 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:33.067060 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-flsvz_41932272-e547-4229-be69-f99a4096637e/insights-operator/0.log" Sep 08 19:00:33.067861 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:33.067826 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-flsvz_41932272-e547-4229-be69-f99a4096637e/insights-operator/1.log" Sep 08 19:00:33.172352 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:33.172322 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rhzfq_29df756b-120f-4026-ac1f-b14507717d42/kube-rbac-proxy/0.log" Sep 08 19:00:33.194784 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:33.194754 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rhzfq_29df756b-120f-4026-ac1f-b14507717d42/exporter/0.log" Sep 08 19:00:33.226644 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:33.226609 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rhzfq_29df756b-120f-4026-ac1f-b14507717d42/extractor/0.log" Sep 08 19:00:33.952273 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:33.952247 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-phw82/perf-node-gather-daemonset-86nrx" Sep 08 19:00:36.025917 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:36.025885 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8d688d88f-l42d7_684038e9-9cd1-457c-b852-d6eabc24e260/manager/0.log" Sep 08 19:00:36.047334 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:36.047310 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8d688d88f-l42d7_684038e9-9cd1-457c-b852-d6eabc24e260/copy-manifests/0.log" Sep 08 19:00:37.365965 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:37.365940 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6b68db98f8-5t66l_8d6c1b59-4982-477e-b4e6-878226168e14/manager/0.log" Sep 08 19:00:37.966760 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:37.966734 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6d985cc74d-b2kg9_7c1b6bd1-08a0-4fce-be8c-648216b13ebe/kube-auth-proxy/0.log" Sep 08 19:00:38.093312 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:38.093283 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7bdcf569fd-2mtjd_c44b4985-58c8-4e47-af99-0fd9052f8c7b/router/0.log" Sep 08 19:00:40.034531 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:40.034440 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6d985cc74d-b2kg9_7c1b6bd1-08a0-4fce-be8c-648216b13ebe/kube-auth-proxy/0.log" Sep 08 19:00:40.153137 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:40.153103 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7bdcf569fd-2mtjd_c44b4985-58c8-4e47-af99-0fd9052f8c7b/router/0.log" Sep 08 19:00:44.133144 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:44.133110 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-8sww7_dedc0ee5-f020-456a-988e-7e4833ed34a6/kube-multus/0.log" Sep 08 19:00:44.524626 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:44.524547 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw886_a9136afd-27d0-4789-925d-0f7934e8f79b/kube-multus-additional-cni-plugins/0.log" Sep 08 19:00:44.547767 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:44.547737 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw886_a9136afd-27d0-4789-925d-0f7934e8f79b/egress-router-binary-copy/0.log" Sep 08 19:00:44.571051 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:44.570994 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw886_a9136afd-27d0-4789-925d-0f7934e8f79b/cni-plugins/0.log" Sep 08 19:00:44.594047 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:44.594020 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw886_a9136afd-27d0-4789-925d-0f7934e8f79b/bond-cni-plugin/0.log" Sep 08 19:00:44.646126 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:44.646088 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw886_a9136afd-27d0-4789-925d-0f7934e8f79b/routeoverride-cni/0.log" Sep 08 19:00:44.670279 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:44.670237 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw886_a9136afd-27d0-4789-925d-0f7934e8f79b/whereabouts-cni-bincopy/0.log" Sep 08 19:00:44.693064 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:44.693036 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vw886_a9136afd-27d0-4789-925d-0f7934e8f79b/whereabouts-cni/0.log" Sep 08 19:00:44.874241 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:44.874208 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-qz9vk_3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2/network-metrics-daemon/0.log" Sep 08 19:00:44.898856 ip-10-0-139-112 kubenswrapper[2555]: I0908 19:00:44.898826 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-qz9vk_3d5d3a1e-2560-49ce-8f2f-c0e8cdde2ec2/kube-rbac-proxy/0.log"