Jul 09 04:30:08.242640 ip-10-0-139-184 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 09 04:30:08.242653 ip-10-0-139-184 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 09 04:30:08.242663 ip-10-0-139-184 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 09 04:30:08.243023 ip-10-0-139-184 systemd[1]: Failed to start Kubernetes Kubelet. Jul 09 04:30:18.291251 ip-10-0-139-184 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 09 04:30:18.291270 ip-10-0-139-184 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot fb4fbfbc4dff484285ff3a92cf9344bd -- Jul 09 04:32:05.985217 ip-10-0-139-184 systemd[1]: Starting Kubernetes Kubelet... Jul 09 04:32:06.349900 ip-10-0-139-184 kubenswrapper[2565]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 04:32:06.349900 ip-10-0-139-184 kubenswrapper[2565]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 09 04:32:06.349900 ip-10-0-139-184 kubenswrapper[2565]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 04:32:06.349900 ip-10-0-139-184 kubenswrapper[2565]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 09 04:32:06.349900 ip-10-0-139-184 kubenswrapper[2565]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 04:32:06.350587 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.349969 2565 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 09 04:32:06.356928 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.356897 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 04:32:06.356928 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.356927 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 04:32:06.356928 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.356932 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 04:32:06.357019 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.356935 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 04:32:06.357019 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.356938 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 04:32:06.357019 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.356941 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 04:32:06.357019 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.356944 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 04:32:06.357019 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.356947 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 04:32:06.357019 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.356950 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 04:32:06.357019 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.356953 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 04:32:06.357019 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.356956 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 04:32:06.357019 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.356962 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 04:32:06.357019 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.356968 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 04:32:06.357019 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.356972 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 04:32:06.357019 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.356975 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 04:32:06.357019 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.356978 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 04:32:06.357019 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.356981 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 04:32:06.357019 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.356984 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 04:32:06.357019 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.356987 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 04:32:06.357019 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.356990 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 04:32:06.357019 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.356993 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 04:32:06.357019 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.356995 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 09 04:32:06.357453 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.356998 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 04:32:06.357453 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357000 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 04:32:06.357453 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357003 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 04:32:06.357453 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357005 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 04:32:06.357453 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357008 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 04:32:06.357453 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357010 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 04:32:06.357453 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357013 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 04:32:06.357453 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357015 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 04:32:06.357453 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357018 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 04:32:06.357453 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357021 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 04:32:06.357453 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357024 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 04:32:06.357453 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357027 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 04:32:06.357453 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357030 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 04:32:06.357453 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357033 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 04:32:06.357453 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357035 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 04:32:06.357453 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357038 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 04:32:06.357453 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357041 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 04:32:06.357453 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357044 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 04:32:06.357453 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357046 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 04:32:06.357453 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357048 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 04:32:06.357971 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357051 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 04:32:06.357971 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357053 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 04:32:06.357971 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357056 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 04:32:06.357971 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357058 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 04:32:06.357971 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357061 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 04:32:06.357971 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357064 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 04:32:06.357971 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357067 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 04:32:06.357971 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357070 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 04:32:06.357971 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357073 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 04:32:06.357971 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357075 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 04:32:06.357971 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357078 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 04:32:06.357971 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357081 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 04:32:06.357971 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357084 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 04:32:06.357971 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357087 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 04:32:06.357971 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357090 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 04:32:06.357971 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357093 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 04:32:06.357971 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357095 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 04:32:06.357971 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357098 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 04:32:06.357971 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357100 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 04:32:06.358452 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357103 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 04:32:06.358452 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357106 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 04:32:06.358452 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357109 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 04:32:06.358452 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357111 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 04:32:06.358452 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357114 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 04:32:06.358452 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357116 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 04:32:06.358452 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357119 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 04:32:06.358452 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357121 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 04:32:06.358452 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357124 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 04:32:06.358452 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357126 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 04:32:06.358452 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357130 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 04:32:06.358452 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357132 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 04:32:06.358452 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357135 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 04:32:06.358452 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357137 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 04:32:06.358452 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357140 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 04:32:06.358452 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357143 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 04:32:06.358452 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357145 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 04:32:06.358452 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357148 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 04:32:06.358452 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357151 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 04:32:06.358452 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357153 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 04:32:06.358940 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357155 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 04:32:06.358940 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357158 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 04:32:06.358940 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357160 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 04:32:06.358940 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357163 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 04:32:06.358940 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357166 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 04:32:06.358940 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357169 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 04:32:06.358940 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357172 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 04:32:06.358940 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357175 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 04:32:06.358940 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357178 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 04:32:06.358940 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357181 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 04:32:06.358940 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357184 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 04:32:06.358940 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357186 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 04:32:06.358940 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357189 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 04:32:06.358940 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357191 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 04:32:06.358940 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357193 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 04:32:06.358940 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357196 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 04:32:06.358940 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357198 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 04:32:06.358940 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357201 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 04:32:06.358940 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357940 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 04:32:06.359392 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357947 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 04:32:06.359392 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357950 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 04:32:06.359392 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357953 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 04:32:06.359392 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357961 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 04:32:06.359392 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357965 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 04:32:06.359392 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357969 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 04:32:06.359392 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357971 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 04:32:06.359392 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357975 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 04:32:06.359392 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357978 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 04:32:06.359392 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357981 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 04:32:06.359392 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357984 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 04:32:06.359392 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357987 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 04:32:06.359392 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357989 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 04:32:06.359392 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357992 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 04:32:06.359392 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357994 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 04:32:06.359392 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.357997 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 04:32:06.359392 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358000 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 04:32:06.359392 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358002 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 04:32:06.359392 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358005 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 04:32:06.359925 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358010 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 04:32:06.359925 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358014 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 04:32:06.359925 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358017 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 04:32:06.359925 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358020 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 04:32:06.359925 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358023 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 04:32:06.359925 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358026 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 04:32:06.359925 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358029 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 04:32:06.359925 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358032 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 04:32:06.359925 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358035 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 09 04:32:06.359925 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358038 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 04:32:06.359925 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358041 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 04:32:06.359925 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358044 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 04:32:06.359925 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358046 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 04:32:06.359925 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358054 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 04:32:06.359925 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358057 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 04:32:06.359925 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358060 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 04:32:06.359925 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358063 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 04:32:06.359925 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358066 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 04:32:06.359925 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358069 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 04:32:06.359925 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358072 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 04:32:06.360441 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358074 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 04:32:06.360441 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358077 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 04:32:06.360441 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358080 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 04:32:06.360441 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358082 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 04:32:06.360441 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358085 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 04:32:06.360441 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358088 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 04:32:06.360441 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358090 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 04:32:06.360441 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358093 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 04:32:06.360441 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358096 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 04:32:06.360441 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358099 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 04:32:06.360441 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358101 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 04:32:06.360441 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358104 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 04:32:06.360441 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358107 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 04:32:06.360441 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358109 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 04:32:06.360441 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358112 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 04:32:06.360441 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358114 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 04:32:06.360441 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358117 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 04:32:06.360441 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358119 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 04:32:06.360441 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358123 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 04:32:06.360441 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358125 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 04:32:06.360934 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358128 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 04:32:06.360934 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358130 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 04:32:06.360934 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358133 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 04:32:06.360934 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358135 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 04:32:06.360934 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358137 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 04:32:06.360934 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358140 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 04:32:06.360934 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358142 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 04:32:06.360934 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358145 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 04:32:06.360934 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358147 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 04:32:06.360934 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358150 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 04:32:06.360934 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358153 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 04:32:06.360934 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358156 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 04:32:06.360934 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358159 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 04:32:06.360934 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358163 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 04:32:06.360934 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358166 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 04:32:06.360934 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358169 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 04:32:06.360934 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358171 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 04:32:06.360934 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358174 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 04:32:06.360934 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358176 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 04:32:06.361398 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358179 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 04:32:06.361398 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358181 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 04:32:06.361398 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358184 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 04:32:06.361398 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358186 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 04:32:06.361398 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358189 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 04:32:06.361398 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358192 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 04:32:06.361398 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358195 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 04:32:06.361398 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358197 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 04:32:06.361398 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358200 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 04:32:06.361398 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358203 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 04:32:06.361398 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358205 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 04:32:06.361398 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358208 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 04:32:06.361398 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358210 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 04:32:06.361398 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358213 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 04:32:06.361398 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358215 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 04:32:06.361398 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358218 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 04:32:06.361398 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358220 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 04:32:06.361398 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358222 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 04:32:06.361398 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358225 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 04:32:06.361398 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358227 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 04:32:06.361876 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358607 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 04:32:06.361876 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358612 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 04:32:06.361876 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358615 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 04:32:06.361876 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358618 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 04:32:06.361876 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358622 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 04:32:06.361876 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358625 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 04:32:06.361876 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358627 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 04:32:06.361876 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358630 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 04:32:06.361876 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358633 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 04:32:06.361876 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358637 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 04:32:06.361876 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358640 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 04:32:06.361876 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358643 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 04:32:06.361876 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358646 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 04:32:06.361876 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358649 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 04:32:06.361876 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358652 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 04:32:06.361876 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358655 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 04:32:06.361876 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358658 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 04:32:06.361876 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358661 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 04:32:06.361876 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358664 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 04:32:06.362351 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358667 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 04:32:06.362351 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358670 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 04:32:06.362351 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358673 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 04:32:06.362351 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358675 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 04:32:06.362351 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358679 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 04:32:06.362351 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358682 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 09 04:32:06.362351 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358684 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 04:32:06.362351 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358687 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 04:32:06.362351 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358689 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 04:32:06.362351 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358692 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 04:32:06.362351 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358694 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 04:32:06.362351 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358696 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 04:32:06.362351 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358699 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 04:32:06.362351 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358702 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 04:32:06.362351 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358704 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 04:32:06.362351 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358706 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 04:32:06.362351 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358709 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 04:32:06.362351 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358711 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 04:32:06.362351 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358714 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 04:32:06.362351 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358716 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 04:32:06.362824 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358719 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 04:32:06.362824 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358721 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 04:32:06.362824 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358724 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 04:32:06.362824 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358726 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 04:32:06.362824 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358729 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 04:32:06.362824 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358731 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 04:32:06.362824 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358734 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 04:32:06.362824 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358737 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 04:32:06.362824 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358739 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 04:32:06.362824 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358742 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 04:32:06.362824 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358745 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 04:32:06.362824 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358747 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 04:32:06.362824 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358750 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 04:32:06.362824 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358752 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 04:32:06.362824 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358755 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 04:32:06.362824 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358757 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 04:32:06.362824 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358760 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 04:32:06.362824 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358762 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 04:32:06.362824 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358765 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 04:32:06.363312 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358767 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 04:32:06.363312 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358769 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 04:32:06.363312 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358772 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 04:32:06.363312 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358775 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 04:32:06.363312 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358779 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 04:32:06.363312 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358781 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 04:32:06.363312 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358784 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 04:32:06.363312 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358786 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 04:32:06.363312 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358789 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 04:32:06.363312 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358793 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 04:32:06.363312 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358796 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 04:32:06.363312 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358798 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 04:32:06.363312 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358801 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 04:32:06.363312 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358806 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 04:32:06.363312 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358808 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 04:32:06.363312 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358811 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 04:32:06.363312 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358814 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 04:32:06.363312 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358816 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 04:32:06.363312 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358819 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 04:32:06.363312 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358821 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 04:32:06.363811 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358824 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 04:32:06.363811 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358826 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 04:32:06.363811 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358829 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 04:32:06.363811 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358832 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 04:32:06.363811 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358836 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 04:32:06.363811 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358839 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 04:32:06.363811 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358841 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 04:32:06.363811 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358844 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 04:32:06.363811 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358847 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 04:32:06.363811 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358850 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 04:32:06.363811 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358853 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 04:32:06.363811 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358855 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 04:32:06.363811 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358858 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 04:32:06.363811 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358860 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 04:32:06.363811 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358863 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 04:32:06.363811 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358865 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 04:32:06.363811 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358868 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 04:32:06.363811 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358870 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 04:32:06.363811 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358873 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 04:32:06.363811 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358875 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 04:32:06.364328 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.358878 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 04:32:06.364328 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.358959 2565 flags.go:64] FLAG: --address="0.0.0.0" Jul 09 04:32:06.364328 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.358975 2565 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 09 04:32:06.364328 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.358982 2565 flags.go:64] FLAG: --anonymous-auth="true" Jul 09 04:32:06.364328 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.358986 2565 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 09 04:32:06.364328 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.358992 2565 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 09 04:32:06.364328 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.358995 2565 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 09 04:32:06.364328 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.358999 2565 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 09 04:32:06.364328 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359004 2565 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 09 04:32:06.364328 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359007 2565 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 09 04:32:06.364328 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359010 2565 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 09 04:32:06.364328 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359013 2565 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 09 04:32:06.364328 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359016 2565 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 09 04:32:06.364328 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359019 2565 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 09 04:32:06.364328 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359022 2565 flags.go:64] FLAG: --cgroup-root="" Jul 09 04:32:06.364328 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359025 2565 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 09 04:32:06.364328 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359028 2565 flags.go:64] FLAG: --client-ca-file="" Jul 09 04:32:06.364328 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359031 2565 flags.go:64] FLAG: --cloud-provider="external" Jul 09 04:32:06.364328 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359034 2565 flags.go:64] FLAG: --cluster-dns="[]" Jul 09 04:32:06.364328 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359038 2565 flags.go:64] FLAG: --cluster-domain="" Jul 09 04:32:06.364328 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359040 2565 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 09 04:32:06.364328 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359043 2565 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 09 04:32:06.364328 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359047 2565 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 09 04:32:06.364328 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359050 2565 flags.go:64] FLAG: --container-log-max-files="5" Jul 09 04:32:06.364882 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359053 2565 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 09 04:32:06.364882 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359056 2565 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 09 04:32:06.364882 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359059 2565 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 09 04:32:06.364882 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359062 2565 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 09 04:32:06.364882 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359065 2565 flags.go:64] FLAG: --contention-profiling="false" Jul 09 04:32:06.364882 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359068 2565 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 09 04:32:06.364882 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359071 2565 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 09 04:32:06.364882 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359074 2565 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 09 04:32:06.364882 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359077 2565 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 09 04:32:06.364882 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359082 2565 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 09 04:32:06.364882 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359085 2565 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 09 04:32:06.364882 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359088 2565 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 09 04:32:06.364882 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359090 2565 flags.go:64] FLAG: --enable-load-reader="false" Jul 09 04:32:06.364882 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359094 2565 flags.go:64] FLAG: --enable-server="true" Jul 09 04:32:06.364882 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359098 2565 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 09 04:32:06.364882 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359103 2565 flags.go:64] FLAG: --event-burst="100" Jul 09 04:32:06.364882 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359106 2565 flags.go:64] FLAG: --event-qps="50" Jul 09 04:32:06.364882 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359109 2565 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 09 04:32:06.364882 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359112 2565 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 09 04:32:06.364882 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359115 2565 flags.go:64] FLAG: --eviction-hard="" Jul 09 04:32:06.364882 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359119 2565 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 09 04:32:06.364882 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359122 2565 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 09 04:32:06.364882 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359125 2565 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 09 04:32:06.364882 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359128 2565 flags.go:64] FLAG: --eviction-soft="" Jul 09 04:32:06.364882 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359131 2565 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 09 04:32:06.365517 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359133 2565 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 09 04:32:06.365517 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359136 2565 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 09 04:32:06.365517 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359139 2565 flags.go:64] FLAG: --experimental-mounter-path="" Jul 09 04:32:06.365517 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359142 2565 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 09 04:32:06.365517 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359145 2565 flags.go:64] FLAG: --fail-swap-on="true" Jul 09 04:32:06.365517 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359148 2565 flags.go:64] FLAG: --feature-gates="" Jul 09 04:32:06.365517 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359152 2565 flags.go:64] FLAG: --file-check-frequency="20s" Jul 09 04:32:06.365517 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359156 2565 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 09 04:32:06.365517 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359159 2565 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 09 04:32:06.365517 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359162 2565 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 09 04:32:06.365517 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359165 2565 flags.go:64] FLAG: --healthz-port="10248" Jul 09 04:32:06.365517 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359168 2565 flags.go:64] FLAG: --help="false" Jul 09 04:32:06.365517 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359171 2565 flags.go:64] FLAG: --hostname-override="ip-10-0-139-184.ec2.internal" Jul 09 04:32:06.365517 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359174 2565 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 09 04:32:06.365517 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359177 2565 flags.go:64] FLAG: --http-check-frequency="20s" Jul 09 04:32:06.365517 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359181 2565 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 09 04:32:06.365517 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359184 2565 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 09 04:32:06.365517 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359187 2565 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 09 04:32:06.365517 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359190 2565 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 09 04:32:06.365517 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359193 2565 flags.go:64] FLAG: --image-service-endpoint="" Jul 09 04:32:06.365517 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359196 2565 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 09 04:32:06.365517 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359199 2565 flags.go:64] FLAG: --kube-api-burst="100" Jul 09 04:32:06.365517 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359202 2565 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 09 04:32:06.366170 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359205 2565 flags.go:64] FLAG: --kube-api-qps="50" Jul 09 04:32:06.366170 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359208 2565 flags.go:64] FLAG: --kube-reserved="" Jul 09 04:32:06.366170 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359211 2565 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 09 04:32:06.366170 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359214 2565 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 09 04:32:06.366170 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359217 2565 flags.go:64] FLAG: --kubelet-cgroups="" Jul 09 04:32:06.366170 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359219 2565 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 09 04:32:06.366170 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359222 2565 flags.go:64] FLAG: --lock-file="" Jul 09 04:32:06.366170 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359225 2565 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 09 04:32:06.366170 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359228 2565 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 09 04:32:06.366170 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359230 2565 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 09 04:32:06.366170 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359235 2565 flags.go:64] FLAG: --log-json-split-stream="false" Jul 09 04:32:06.366170 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359238 2565 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 09 04:32:06.366170 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359241 2565 flags.go:64] FLAG: --log-text-split-stream="false" Jul 09 04:32:06.366170 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359246 2565 flags.go:64] FLAG: --logging-format="text" Jul 09 04:32:06.366170 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359249 2565 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 09 04:32:06.366170 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359252 2565 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 09 04:32:06.366170 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359255 2565 flags.go:64] FLAG: --manifest-url="" Jul 09 04:32:06.366170 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359258 2565 flags.go:64] FLAG: --manifest-url-header="" Jul 09 04:32:06.366170 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359262 2565 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 09 04:32:06.366170 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359265 2565 flags.go:64] FLAG: --max-open-files="1000000" Jul 09 04:32:06.366170 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359269 2565 flags.go:64] FLAG: --max-pods="110" Jul 09 04:32:06.366170 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359272 2565 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 09 04:32:06.366170 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359275 2565 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 09 04:32:06.366170 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359278 2565 flags.go:64] FLAG: --memory-manager-policy="None" Jul 09 04:32:06.366170 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359281 2565 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 09 04:32:06.366777 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359284 2565 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 09 04:32:06.366777 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359287 2565 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 09 04:32:06.366777 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359290 2565 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 09 04:32:06.366777 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359296 2565 flags.go:64] FLAG: --node-status-max-images="50" Jul 09 04:32:06.366777 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359299 2565 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 09 04:32:06.366777 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359302 2565 flags.go:64] FLAG: --oom-score-adj="-999" Jul 09 04:32:06.366777 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359305 2565 flags.go:64] FLAG: --pod-cidr="" Jul 09 04:32:06.366777 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359308 2565 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 09 04:32:06.366777 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359313 2565 flags.go:64] FLAG: --pod-manifest-path="" Jul 09 04:32:06.366777 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359316 2565 flags.go:64] FLAG: --pod-max-pids="-1" Jul 09 04:32:06.366777 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359319 2565 flags.go:64] FLAG: --pods-per-core="0" Jul 09 04:32:06.366777 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359322 2565 flags.go:64] FLAG: --port="10250" Jul 09 04:32:06.366777 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359324 2565 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 09 04:32:06.366777 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359327 2565 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0218b486410e9fa54" Jul 09 04:32:06.366777 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359330 2565 flags.go:64] FLAG: --qos-reserved="" Jul 09 04:32:06.366777 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359333 2565 flags.go:64] FLAG: --read-only-port="10255" Jul 09 04:32:06.366777 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359336 2565 flags.go:64] FLAG: --register-node="true" Jul 09 04:32:06.366777 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359339 2565 flags.go:64] FLAG: --register-with-taints="" Jul 09 04:32:06.366777 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359342 2565 flags.go:64] FLAG: --registry-burst="10" Jul 09 04:32:06.366777 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359345 2565 flags.go:64] FLAG: --registry-qps="5" Jul 09 04:32:06.366777 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359350 2565 flags.go:64] FLAG: --reserved-cpus="" Jul 09 04:32:06.366777 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359352 2565 flags.go:64] FLAG: --reserved-memory="" Jul 09 04:32:06.366777 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359356 2565 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 09 04:32:06.366777 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359359 2565 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 09 04:32:06.366777 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359362 2565 flags.go:64] FLAG: --rotate-certificates="false" Jul 09 04:32:06.367395 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359365 2565 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 09 04:32:06.367395 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359367 2565 flags.go:64] FLAG: --runonce="false" Jul 09 04:32:06.367395 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359370 2565 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 09 04:32:06.367395 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359373 2565 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 09 04:32:06.367395 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359376 2565 flags.go:64] FLAG: --seccomp-default="false" Jul 09 04:32:06.367395 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359379 2565 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 09 04:32:06.367395 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359382 2565 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 09 04:32:06.367395 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359385 2565 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 09 04:32:06.367395 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359388 2565 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 09 04:32:06.367395 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359391 2565 flags.go:64] FLAG: --storage-driver-password="root" Jul 09 04:32:06.367395 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359394 2565 flags.go:64] FLAG: --storage-driver-secure="false" Jul 09 04:32:06.367395 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359397 2565 flags.go:64] FLAG: --storage-driver-table="stats" Jul 09 04:32:06.367395 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359401 2565 flags.go:64] FLAG: --storage-driver-user="root" Jul 09 04:32:06.367395 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359405 2565 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 09 04:32:06.367395 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359408 2565 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 09 04:32:06.367395 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359412 2565 flags.go:64] FLAG: --system-cgroups="" Jul 09 04:32:06.367395 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359415 2565 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 09 04:32:06.367395 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359420 2565 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 09 04:32:06.367395 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359423 2565 flags.go:64] FLAG: --tls-cert-file="" Jul 09 04:32:06.367395 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359426 2565 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 09 04:32:06.367395 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359982 2565 flags.go:64] FLAG: --tls-min-version="" Jul 09 04:32:06.367395 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359986 2565 flags.go:64] FLAG: --tls-private-key-file="" Jul 09 04:32:06.367395 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359989 2565 flags.go:64] FLAG: --topology-manager-policy="none" Jul 09 04:32:06.367395 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359991 2565 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 09 04:32:06.367395 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359994 2565 flags.go:64] FLAG: --topology-manager-scope="container" Jul 09 04:32:06.368070 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.359998 2565 flags.go:64] FLAG: --v="2" Jul 09 04:32:06.368070 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.360002 2565 flags.go:64] FLAG: --version="false" Jul 09 04:32:06.368070 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.360008 2565 flags.go:64] FLAG: --vmodule="" Jul 09 04:32:06.368070 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.360012 2565 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 09 04:32:06.368070 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.360015 2565 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 09 04:32:06.368070 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360136 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 04:32:06.368070 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360140 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 04:32:06.368070 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360143 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 04:32:06.368070 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360146 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 04:32:06.368070 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360150 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 04:32:06.368070 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360153 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 04:32:06.368070 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360155 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 04:32:06.368070 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360158 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 04:32:06.368070 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360160 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 04:32:06.368070 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360163 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 04:32:06.368070 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360165 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 04:32:06.368070 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360168 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 04:32:06.368070 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360170 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 04:32:06.368070 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360173 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 04:32:06.368070 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360177 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 04:32:06.368070 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360179 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 04:32:06.368761 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360182 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 04:32:06.368761 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360184 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 04:32:06.368761 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360187 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 04:32:06.368761 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360190 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 04:32:06.368761 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360193 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 04:32:06.368761 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360195 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 04:32:06.368761 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360198 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 04:32:06.368761 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360200 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 04:32:06.368761 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360203 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 04:32:06.368761 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360205 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 04:32:06.368761 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360209 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 04:32:06.368761 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360213 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 04:32:06.368761 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360216 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 04:32:06.368761 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360223 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 09 04:32:06.368761 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360226 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 04:32:06.368761 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360229 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 04:32:06.368761 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360232 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 04:32:06.368761 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360234 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 04:32:06.368761 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360237 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 04:32:06.368761 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360240 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 04:32:06.369579 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360242 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 04:32:06.369579 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360245 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 04:32:06.369579 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360248 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 04:32:06.369579 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360250 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 04:32:06.369579 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360253 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 04:32:06.369579 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360255 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 04:32:06.369579 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360258 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 04:32:06.369579 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360261 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 04:32:06.369579 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360263 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 04:32:06.369579 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360266 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 04:32:06.369579 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360269 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 04:32:06.369579 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360272 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 04:32:06.369579 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360275 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 04:32:06.369579 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360277 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 04:32:06.369579 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360280 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 04:32:06.369579 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360283 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 04:32:06.369579 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360285 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 04:32:06.369579 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360288 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 04:32:06.369579 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360291 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 04:32:06.369579 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360293 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 04:32:06.370248 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360296 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 04:32:06.370248 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360300 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 04:32:06.370248 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360303 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 04:32:06.370248 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360306 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 04:32:06.370248 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360309 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 04:32:06.370248 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360312 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 04:32:06.370248 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360315 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 04:32:06.370248 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360318 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 04:32:06.370248 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360321 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 04:32:06.370248 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360324 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 04:32:06.370248 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360326 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 04:32:06.370248 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360329 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 04:32:06.370248 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360332 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 04:32:06.370248 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360334 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 04:32:06.370248 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360336 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 04:32:06.370248 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360339 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 04:32:06.370248 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360343 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 04:32:06.370248 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360347 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 04:32:06.370248 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360349 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 04:32:06.370737 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360352 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 04:32:06.370737 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360354 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 04:32:06.370737 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360356 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 04:32:06.370737 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360361 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 04:32:06.370737 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360363 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 04:32:06.370737 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360366 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 04:32:06.370737 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360369 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 04:32:06.370737 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360371 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 04:32:06.370737 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360374 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 04:32:06.370737 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360377 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 04:32:06.370737 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360380 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 04:32:06.370737 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360382 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 04:32:06.370737 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360385 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 04:32:06.370737 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360387 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 04:32:06.370737 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360390 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 04:32:06.370737 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360393 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 04:32:06.370737 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360395 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 04:32:06.370737 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360398 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 04:32:06.370737 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360403 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 04:32:06.370737 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360405 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 04:32:06.371239 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360408 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 04:32:06.371239 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360410 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 04:32:06.371239 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360413 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 04:32:06.371239 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.360415 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 04:32:06.371239 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.360421 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 04:32:06.371239 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.367367 2565 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 09 04:32:06.371239 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.367386 2565 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 09 04:32:06.371239 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.367402 2565 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 09 04:32:06.371239 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.367409 2565 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 09 04:32:06.371239 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367469 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 04:32:06.371239 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367474 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 04:32:06.371239 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367477 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 04:32:06.371239 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367481 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 04:32:06.371239 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367484 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 04:32:06.371239 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367487 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 04:32:06.371659 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367489 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 04:32:06.371659 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367493 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 04:32:06.371659 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367496 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 04:32:06.371659 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367499 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 04:32:06.371659 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367501 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 04:32:06.371659 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367504 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 04:32:06.371659 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367507 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 04:32:06.371659 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367510 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 04:32:06.371659 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367514 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 04:32:06.371659 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367518 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 04:32:06.371659 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367523 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 04:32:06.371659 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367528 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 04:32:06.371659 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367533 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 04:32:06.371659 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367537 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 04:32:06.371659 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367540 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 04:32:06.371659 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367543 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 04:32:06.371659 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367545 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 04:32:06.371659 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367548 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 04:32:06.371659 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367550 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 04:32:06.372140 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367553 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 04:32:06.372140 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367556 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 04:32:06.372140 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367559 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 04:32:06.372140 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367561 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 04:32:06.372140 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367573 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 04:32:06.372140 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367576 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 04:32:06.372140 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367579 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 04:32:06.372140 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367582 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 04:32:06.372140 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367584 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 04:32:06.372140 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367587 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 04:32:06.372140 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367590 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 04:32:06.372140 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367592 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 04:32:06.372140 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367595 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 04:32:06.372140 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367597 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 04:32:06.372140 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367601 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 04:32:06.372140 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367605 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 04:32:06.372140 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367609 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 04:32:06.372140 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367615 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 04:32:06.372140 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367618 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 04:32:06.372596 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367620 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 04:32:06.372596 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367623 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 04:32:06.372596 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367626 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 04:32:06.372596 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367629 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 04:32:06.372596 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367631 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 04:32:06.372596 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367634 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 04:32:06.372596 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367636 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 04:32:06.372596 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367639 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 04:32:06.372596 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367641 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 04:32:06.372596 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367644 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 04:32:06.372596 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367646 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 04:32:06.372596 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367649 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 04:32:06.372596 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367652 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 04:32:06.372596 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367654 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 04:32:06.372596 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367657 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 04:32:06.372596 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367659 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 04:32:06.372596 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367662 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 09 04:32:06.372596 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367665 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 04:32:06.372596 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367669 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 04:32:06.372596 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367671 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 04:32:06.373193 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367674 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 04:32:06.373193 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367677 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 04:32:06.373193 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367681 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 04:32:06.373193 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367686 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 04:32:06.373193 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367690 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 04:32:06.373193 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367692 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 04:32:06.373193 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367695 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 04:32:06.373193 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367698 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 04:32:06.373193 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367700 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 04:32:06.373193 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367703 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 04:32:06.373193 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367705 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 04:32:06.373193 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367708 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 04:32:06.373193 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367710 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 04:32:06.373193 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367713 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 04:32:06.373193 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367716 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 04:32:06.373193 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367718 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 04:32:06.373193 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367721 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 04:32:06.373193 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367723 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 04:32:06.373193 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367726 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 04:32:06.373193 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367729 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 04:32:06.373679 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367731 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 04:32:06.373679 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367734 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 04:32:06.373679 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367736 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 04:32:06.373679 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367738 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 04:32:06.373679 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367741 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 04:32:06.373679 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367743 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 04:32:06.373679 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367746 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 04:32:06.373679 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367748 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 04:32:06.373679 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367751 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 04:32:06.373679 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367755 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 04:32:06.373679 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367759 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 04:32:06.373679 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367764 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 04:32:06.373679 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367768 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 04:32:06.373679 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367771 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 04:32:06.373679 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367774 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 04:32:06.374096 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.367779 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 04:32:06.374096 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367921 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 04:32:06.374096 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367929 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 04:32:06.374096 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367932 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 04:32:06.374096 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367936 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 04:32:06.374096 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367940 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 04:32:06.374096 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367943 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 04:32:06.374096 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367946 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 04:32:06.374096 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367949 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 04:32:06.374096 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367952 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 04:32:06.374096 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367955 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 04:32:06.374096 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367957 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 04:32:06.374096 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367960 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 04:32:06.374096 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367963 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 04:32:06.374433 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367965 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 04:32:06.374433 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367968 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 04:32:06.374433 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367970 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 04:32:06.374433 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367973 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 04:32:06.374433 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367975 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 04:32:06.374433 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367978 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 04:32:06.374433 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367980 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 04:32:06.374433 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367983 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 04:32:06.374433 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367985 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 04:32:06.374433 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367988 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 04:32:06.374433 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367991 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 04:32:06.374433 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367995 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 04:32:06.374433 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.367999 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 04:32:06.374433 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368004 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 04:32:06.374433 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368007 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 04:32:06.374433 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368011 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 04:32:06.374433 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368014 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 04:32:06.374433 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368016 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 04:32:06.374433 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368019 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 04:32:06.374433 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368022 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 04:32:06.374927 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368024 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 04:32:06.374927 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368027 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 04:32:06.374927 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368029 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 04:32:06.374927 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368033 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 04:32:06.374927 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368036 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 04:32:06.374927 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368038 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 04:32:06.374927 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368041 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 04:32:06.374927 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368043 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 04:32:06.374927 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368045 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 04:32:06.374927 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368048 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 04:32:06.374927 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368051 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 04:32:06.374927 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368053 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 04:32:06.374927 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368056 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 04:32:06.374927 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368058 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 04:32:06.374927 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368061 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 04:32:06.374927 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368063 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 04:32:06.374927 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368066 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 04:32:06.374927 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368071 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 04:32:06.374927 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368075 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 04:32:06.375391 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368079 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 04:32:06.375391 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368084 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 04:32:06.375391 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368087 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 04:32:06.375391 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368090 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 04:32:06.375391 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368094 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 04:32:06.375391 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368096 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 04:32:06.375391 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368099 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 04:32:06.375391 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368102 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 04:32:06.375391 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368104 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 04:32:06.375391 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368108 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 04:32:06.375391 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368110 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 04:32:06.375391 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368113 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 04:32:06.375391 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368115 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 04:32:06.375391 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368118 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 04:32:06.375391 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368120 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 04:32:06.375391 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368123 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 04:32:06.375391 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368125 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 04:32:06.375391 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368127 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 04:32:06.375391 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368130 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 04:32:06.375391 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368133 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 04:32:06.375869 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368135 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 04:32:06.375869 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368138 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 04:32:06.375869 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368140 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 04:32:06.375869 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368143 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 04:32:06.375869 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368147 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 04:32:06.375869 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368151 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 04:32:06.375869 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368155 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 09 04:32:06.375869 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368159 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 04:32:06.375869 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368162 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 04:32:06.375869 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368164 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 04:32:06.375869 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368167 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 04:32:06.375869 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368169 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 04:32:06.375869 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368171 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 04:32:06.375869 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368174 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 04:32:06.375869 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368177 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 04:32:06.375869 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368179 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 04:32:06.375869 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368182 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 04:32:06.375869 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368186 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 04:32:06.375869 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368189 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 04:32:06.375869 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368191 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 04:32:06.376379 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368194 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 04:32:06.376379 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368196 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 04:32:06.376379 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368199 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 04:32:06.376379 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368201 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 04:32:06.376379 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368204 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 04:32:06.376379 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368207 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 04:32:06.376379 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:06.368209 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 04:32:06.376379 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.368215 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 04:32:06.376379 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.369160 2565 server.go:962] "Client rotation is on, will bootstrap in background" Jul 09 04:32:06.376379 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.371678 2565 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 09 04:32:06.376379 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.373268 2565 server.go:1019] "Starting client certificate rotation" Jul 09 04:32:06.376379 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.373363 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 04:32:06.376379 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.373399 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 04:32:06.376736 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.375715 2565 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 04:32:06.376975 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.376893 2565 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 04:32:06.389938 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.389902 2565 log.go:25] "Validated CRI v1 runtime API" Jul 09 04:32:06.392886 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.392872 2565 log.go:25] "Validated CRI v1 image API" Jul 09 04:32:06.394498 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.394438 2565 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 09 04:32:06.397380 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.397352 2565 fs.go:135] Filesystem UUIDs: map[04f941fe-86de-40bf-9fc9-738b06947001:/dev/nvme0n1p4 053b4603-634a-46b8-bbda-9dbdf4e7f622:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 09 04:32:06.397447 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.397379 2565 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 09 04:32:06.400373 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.400356 2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 04:32:06.402743 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.402631 2565 manager.go:216] Machine: {Timestamp:2026-07-09 04:32:06.400897438 +0000 UTC m=+0.303852857 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097491 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27a48246ef878d0181009aadc6a5df SystemUUID:ec27a482-46ef-878d-0181-009aadc6a5df BootID:fb4fbfbc-4dff-4842-85ff-3a92cf9344bd Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:2d:b1:d1:0a:6b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:2d:b1:d1:0a:6b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:86:cf:0e:8c:27:5e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 09 04:32:06.402743 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.402741 2565 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 09 04:32:06.402871 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.402860 2565 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 09 04:32:06.403799 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.403777 2565 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 09 04:32:06.403939 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.403800 2565 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-184.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 09 04:32:06.404540 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.404531 2565 topology_manager.go:138] "Creating topology manager with none policy" Jul 09 04:32:06.404576 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.404542 2565 container_manager_linux.go:309] "Creating device plugin manager" Jul 09 04:32:06.404576 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.404549 2565 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 04:32:06.405241 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.405231 2565 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 04:32:06.406992 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.406982 2565 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 09 04:32:06.410375 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.410363 2565 state_mem.go:36] "Initialized new in-memory state store" Jul 09 04:32:06.410479 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.410469 2565 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 09 04:32:06.410510 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.410498 2565 kubelet.go:486] "Attempting to sync node with API server" Jul 09 04:32:06.410542 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.410515 2565 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 09 04:32:06.410542 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.410532 2565 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 09 04:32:06.410594 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.410547 2565 kubelet.go:398] "Adding apiserver pod source" Jul 09 04:32:06.410594 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.410558 2565 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 09 04:32:06.410928 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.410902 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 04:32:06.411013 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.410939 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 04:32:06.414446 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.414315 2565 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 09 04:32:06.416886 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.416862 2565 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 09 04:32:06.416971 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.416895 2565 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 09 04:32:06.418975 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.418960 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 09 04:32:06.418975 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.418977 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 09 04:32:06.419081 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.418984 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 09 04:32:06.419081 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.418990 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 09 04:32:06.419081 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.418998 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 09 04:32:06.419081 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.419003 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 09 04:32:06.419081 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.419011 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 09 04:32:06.419081 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.419016 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 09 04:32:06.419081 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.419023 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 09 04:32:06.419081 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.419028 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 09 04:32:06.419081 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.419036 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 09 04:32:06.419081 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.419041 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 09 04:32:06.419617 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.419608 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 09 04:32:06.419617 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.419616 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 09 04:32:06.420194 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.420174 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-957f4" Jul 09 04:32:06.421009 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:06.420982 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-184.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 09 04:32:06.421166 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:06.421146 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 09 04:32:06.422866 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.422854 2565 server.go:1268] "Started kubelet" Jul 09 04:32:06.422954 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.422926 2565 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 09 04:32:06.423153 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.422928 2565 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 09 04:32:06.423262 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.423185 2565 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 09 04:32:06.423762 ip-10-0-139-184 systemd[1]: Started Kubernetes Kubelet. Jul 09 04:32:06.424860 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.424846 2565 server.go:310] "Adding debug handlers to kubelet server" Jul 09 04:32:06.427860 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.427842 2565 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 09 04:32:06.428271 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.428251 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-957f4" Jul 09 04:32:06.428350 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:06.428286 2565 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 09 04:32:06.429936 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.429920 2565 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-184.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 09 04:32:06.430785 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:06.429969 2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-184.ec2.internal.18c084da7fef0cf6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-184.ec2.internal,UID:ip-10-0-139-184.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-184.ec2.internal,},FirstTimestamp:2026-07-09 04:32:06.42282623 +0000 UTC m=+0.325781644,LastTimestamp:2026-07-09 04:32:06.42282623 +0000 UTC m=+0.325781644,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-184.ec2.internal,}" Jul 09 04:32:06.434866 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.434851 2565 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 09 04:32:06.434951 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.434861 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 09 04:32:06.435012 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:06.434975 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-184.ec2.internal\" not found" Jul 09 04:32:06.435404 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.435382 2565 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 09 04:32:06.435464 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.435385 2565 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 09 04:32:06.435464 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.435418 2565 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 09 04:32:06.435554 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.435543 2565 reconstruct.go:97] "Volume reconstruction finished" Jul 09 04:32:06.435609 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.435556 2565 reconciler.go:29] "Reconciler: start to sync state" Jul 09 04:32:06.435740 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.435723 2565 factory.go:55] Registering systemd factory Jul 09 04:32:06.435740 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.435741 2565 factory.go:223] Registration of the systemd container factory successfully Jul 09 04:32:06.435950 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.435931 2565 factory.go:153] Registering CRI-O factory Jul 09 04:32:06.435950 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.435947 2565 factory.go:223] Registration of the crio container factory successfully Jul 09 04:32:06.436065 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.436000 2565 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 09 04:32:06.436065 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.436025 2565 factory.go:103] Registering Raw factory Jul 09 04:32:06.436065 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.436041 2565 manager.go:1196] Started watching for new ooms in manager Jul 09 04:32:06.436595 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.436459 2565 manager.go:349] Starting recovery of all containers Jul 09 04:32:06.437369 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:06.437229 2565 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-184.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 09 04:32:06.437470 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:06.437442 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 09 04:32:06.443752 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.443714 2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 09 04:32:06.447177 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.447153 2565 manager.go:354] Recovery completed Jul 09 04:32:06.453338 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.453323 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 04:32:06.455323 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.455309 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-184.ec2.internal" event="NodeHasSufficientMemory" Jul 09 04:32:06.455373 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.455338 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-184.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 04:32:06.455373 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.455348 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-184.ec2.internal" event="NodeHasSufficientPID" Jul 09 04:32:06.455851 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.455831 2565 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 09 04:32:06.455851 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.455850 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 04:32:06.456018 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.455849 2565 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 09 04:32:06.456018 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.455898 2565 state_mem.go:36] "Initialized new in-memory state store" Jul 09 04:32:06.456518 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.456503 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-184.ec2.internal" event="NodeHasSufficientMemory" Jul 09 04:32:06.456576 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.456527 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-184.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 04:32:06.456576 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.456538 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-184.ec2.internal" event="NodeHasSufficientPID" Jul 09 04:32:06.457041 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.457030 2565 policy_none.go:49] "None policy: Start" Jul 09 04:32:06.457104 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.457044 2565 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 09 04:32:06.457104 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.457055 2565 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 09 04:32:06.457962 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:06.457951 2565 policy_none.go:47] "Start" Jul 09 04:32:08.147200 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.147027 2565 manager.go:337] "Starting Device Plugin manager" Jul 09 04:32:08.147299 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:08.147223 2565 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 09 04:32:08.147299 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.147238 2565 server.go:87] "Starting device plugin registration server" Jul 09 04:32:08.147512 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.147500 2565 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 09 04:32:08.147567 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.147513 2565 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 09 04:32:08.147635 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.147615 2565 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 09 04:32:08.147736 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.147698 2565 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 09 04:32:08.147736 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.147709 2565 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 09 04:32:08.148220 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:08.148205 2565 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 09 04:32:08.148296 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:08.148247 2565 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-184.ec2.internal\" not found" Jul 09 04:32:08.188941 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.188916 2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 09 04:32:08.189861 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.188956 2565 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 09 04:32:08.189861 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.188982 2565 kubelet.go:2448] "Starting kubelet main sync loop" Jul 09 04:32:08.189861 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:08.189019 2565 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 09 04:32:08.191352 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.191323 2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 04:32:08.248289 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.248256 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 04:32:08.249232 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.249217 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-184.ec2.internal" event="NodeHasSufficientMemory" Jul 09 04:32:08.249324 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.249248 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-184.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 04:32:08.249324 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.249259 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-184.ec2.internal" event="NodeHasSufficientPID" Jul 09 04:32:08.249324 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.249287 2565 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-184.ec2.internal" Jul 09 04:32:08.255242 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.255187 2565 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-184.ec2.internal" Jul 09 04:32:08.255242 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:08.255212 2565 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-184.ec2.internal\": node \"ip-10-0-139-184.ec2.internal\" not found" Jul 09 04:32:08.268592 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:08.268564 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-184.ec2.internal\" not found" Jul 09 04:32:08.289291 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.289263 2565 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-184.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-184.ec2.internal"] Jul 09 04:32:08.289403 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.289332 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 04:32:08.290374 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.290358 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-184.ec2.internal" event="NodeHasSufficientMemory" Jul 09 04:32:08.290446 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.290388 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-184.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 04:32:08.290446 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.290400 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-184.ec2.internal" event="NodeHasSufficientPID" Jul 09 04:32:08.291591 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.291578 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 04:32:08.291675 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.291657 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-184.ec2.internal" Jul 09 04:32:08.291708 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.291696 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 04:32:08.292549 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.292532 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-184.ec2.internal" event="NodeHasSufficientMemory" Jul 09 04:32:08.292597 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.292566 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-184.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 04:32:08.292597 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.292532 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-184.ec2.internal" event="NodeHasSufficientMemory" Jul 09 04:32:08.292668 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.292603 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-184.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 04:32:08.292668 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.292615 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-184.ec2.internal" event="NodeHasSufficientPID" Jul 09 04:32:08.292668 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.292576 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-184.ec2.internal" event="NodeHasSufficientPID" Jul 09 04:32:08.293835 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.293820 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-184.ec2.internal" Jul 09 04:32:08.293877 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.293849 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 04:32:08.294583 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.294566 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-184.ec2.internal" event="NodeHasSufficientMemory" Jul 09 04:32:08.294661 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.294596 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-184.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 04:32:08.294661 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.294611 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-184.ec2.internal" event="NodeHasSufficientPID" Jul 09 04:32:08.312913 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:08.312892 2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-184.ec2.internal\" not found" node="ip-10-0-139-184.ec2.internal" Jul 09 04:32:08.316923 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:08.316904 2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-184.ec2.internal\" not found" node="ip-10-0-139-184.ec2.internal" Jul 09 04:32:08.369203 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:08.369170 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-184.ec2.internal\" not found" Jul 09 04:32:08.389078 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.389043 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4e5b88752c85b426292efed1a2dab6a2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-184.ec2.internal\" (UID: \"4e5b88752c85b426292efed1a2dab6a2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-184.ec2.internal" Jul 09 04:32:08.389078 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.389076 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e5b88752c85b426292efed1a2dab6a2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-184.ec2.internal\" (UID: \"4e5b88752c85b426292efed1a2dab6a2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-184.ec2.internal" Jul 09 04:32:08.389295 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.389097 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/be23e120c66897ba7a1764c4c10c1b91-config\") pod \"kube-apiserver-proxy-ip-10-0-139-184.ec2.internal\" (UID: \"be23e120c66897ba7a1764c4c10c1b91\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-184.ec2.internal" Jul 09 04:32:08.469523 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:08.469491 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-184.ec2.internal\" not found" Jul 09 04:32:08.489914 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.489886 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4e5b88752c85b426292efed1a2dab6a2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-184.ec2.internal\" (UID: \"4e5b88752c85b426292efed1a2dab6a2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-184.ec2.internal" Jul 09 04:32:08.490035 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.489925 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e5b88752c85b426292efed1a2dab6a2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-184.ec2.internal\" (UID: \"4e5b88752c85b426292efed1a2dab6a2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-184.ec2.internal" Jul 09 04:32:08.490035 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.489952 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/be23e120c66897ba7a1764c4c10c1b91-config\") pod \"kube-apiserver-proxy-ip-10-0-139-184.ec2.internal\" (UID: \"be23e120c66897ba7a1764c4c10c1b91\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-184.ec2.internal" Jul 09 04:32:08.490035 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.489985 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4e5b88752c85b426292efed1a2dab6a2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-184.ec2.internal\" (UID: \"4e5b88752c85b426292efed1a2dab6a2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-184.ec2.internal" Jul 09 04:32:08.490035 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.490017 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e5b88752c85b426292efed1a2dab6a2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-184.ec2.internal\" (UID: \"4e5b88752c85b426292efed1a2dab6a2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-184.ec2.internal" Jul 09 04:32:08.490035 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.490027 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/be23e120c66897ba7a1764c4c10c1b91-config\") pod \"kube-apiserver-proxy-ip-10-0-139-184.ec2.internal\" (UID: \"be23e120c66897ba7a1764c4c10c1b91\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-184.ec2.internal" Jul 09 04:32:08.570421 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:08.570329 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-184.ec2.internal\" not found" Jul 09 04:32:08.616842 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.616806 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-184.ec2.internal" Jul 09 04:32:08.621029 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:08.620935 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-184.ec2.internal" Jul 09 04:32:08.671190 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:08.671154 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-184.ec2.internal\" not found" Jul 09 04:32:08.771689 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:08.771658 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-184.ec2.internal\" not found" Jul 09 04:32:08.872285 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:08.872211 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-184.ec2.internal\" not found" Jul 09 04:32:08.972839 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:08.972810 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-184.ec2.internal\" not found" Jul 09 04:32:09.008962 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:09.008933 2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 04:32:09.024281 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:09.024250 2565 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 09 04:32:09.024463 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:09.024424 2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 04:32:09.024463 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:09.024424 2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 04:32:09.073796 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:09.073765 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-184.ec2.internal\" not found" Jul 09 04:32:09.085612 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:09.085568 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 04:27:08 +0000 UTC" deadline="2028-03-23 14:54:55.703332096 +0000 UTC" Jul 09 04:32:09.085612 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:09.085606 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14962h22m46.617729193s" Jul 09 04:32:09.174409 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:09.174377 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-184.ec2.internal\" not found" Jul 09 04:32:09.243462 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:09.243430 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe23e120c66897ba7a1764c4c10c1b91.slice/crio-317a66ea17d1b245a4df2168e0eee51ee92dcbf8b2f901c2c6fbf29cc3785dc9 WatchSource:0}: Error finding container 317a66ea17d1b245a4df2168e0eee51ee92dcbf8b2f901c2c6fbf29cc3785dc9: Status 404 returned error can't find the container with id 317a66ea17d1b245a4df2168e0eee51ee92dcbf8b2f901c2c6fbf29cc3785dc9 Jul 09 04:32:09.247583 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:09.247568 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 04:32:09.274904 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:09.274875 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-184.ec2.internal\" not found" Jul 09 04:32:09.281517 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:09.281496 2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 04:32:09.363096 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:09.363069 2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 04:32:09.387149 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:09.387124 2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-184.ec2.internal" Jul 09 04:32:09.397782 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:09.397761 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 04:32:09.398661 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:09.398647 2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-184.ec2.internal" Jul 09 04:32:09.406656 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:09.406640 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 04:32:09.493944 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:09.493920 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e5b88752c85b426292efed1a2dab6a2.slice/crio-055c57fd884490d5cf11890184c54fca8b8aaa43f91c9bf6a1171a8ad4367850 WatchSource:0}: Error finding container 055c57fd884490d5cf11890184c54fca8b8aaa43f91c9bf6a1171a8ad4367850: Status 404 returned error can't find the container with id 055c57fd884490d5cf11890184c54fca8b8aaa43f91c9bf6a1171a8ad4367850 Jul 09 04:32:10.000019 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:09.999989 2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 04:32:10.063133 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.063097 2565 apiserver.go:52] "Watching apiserver" Jul 09 04:32:10.071074 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.071048 2565 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 09 04:32:10.071580 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.071557 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-2tgfm","openshift-multus/network-metrics-daemon-gpx47","openshift-network-operator/iptables-alerter-bzk6k","openshift-ovn-kubernetes/ovnkube-node-t85dg","kube-system/global-pull-secret-syncer-zl4tp","kube-system/kube-apiserver-proxy-ip-10-0-139-184.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g7j6","openshift-image-registry/node-ca-j7zhv","openshift-multus/multus-additional-cni-plugins-cmpgg","openshift-network-diagnostics/network-check-target-bgsl7","kube-system/konnectivity-agent-fltwp","openshift-cluster-node-tuning-operator/tuned-vkz4b","openshift-dns/node-resolver-xzgbm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-184.ec2.internal"] Jul 09 04:32:10.073076 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.073053 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g7j6" Jul 09 04:32:10.074284 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.074261 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xzgbm" Jul 09 04:32:10.075541 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.075515 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-8lkvv\"" Jul 09 04:32:10.075644 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.075609 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 09 04:32:10.075705 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.075696 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 09 04:32:10.075844 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.075825 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 09 04:32:10.075908 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.075892 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bzk6k" Jul 09 04:32:10.076594 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.076569 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9rcs2\"" Jul 09 04:32:10.076712 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.076624 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 09 04:32:10.076712 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.076655 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 09 04:32:10.077077 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.077055 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.078565 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.078305 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 09 04:32:10.078565 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.078352 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 09 04:32:10.078565 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.078501 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 09 04:32:10.078565 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.078545 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-lgtgv\"" Jul 09 04:32:10.078791 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.078677 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zl4tp" Jul 09 04:32:10.078791 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:10.078733 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zl4tp" podUID="d0a12812-bbc1-4b88-a2f2-858d22d67e64" Jul 09 04:32:10.079553 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.079534 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 09 04:32:10.079659 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.079541 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 09 04:32:10.079709 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.079665 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-j7zhv" Jul 09 04:32:10.079764 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.079750 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 09 04:32:10.079815 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.079761 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 09 04:32:10.080410 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.080327 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 09 04:32:10.080499 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.080410 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 09 04:32:10.080499 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.080386 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bqtmb\"" Jul 09 04:32:10.081064 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.081046 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cmpgg" Jul 09 04:32:10.081785 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.081759 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-w2fr6\"" Jul 09 04:32:10.082227 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.082141 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 09 04:32:10.082227 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.082164 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 09 04:32:10.082633 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.082517 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.083213 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.082974 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 09 04:32:10.083550 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.083531 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 09 04:32:10.083874 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.083745 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 09 04:32:10.083874 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.083748 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 09 04:32:10.083874 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.083787 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 09 04:32:10.083874 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.083846 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 09 04:32:10.084115 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.083976 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-m4kmm\"" Jul 09 04:32:10.084115 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.084036 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpx47" Jul 09 04:32:10.084223 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:10.084118 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpx47" podUID="dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa" Jul 09 04:32:10.084788 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.084771 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-mscgq\"" Jul 09 04:32:10.084883 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.084820 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 09 04:32:10.085434 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.085418 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bgsl7" Jul 09 04:32:10.085510 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:10.085474 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bgsl7" podUID="f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59" Jul 09 04:32:10.086780 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.086765 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 09 04:32:10.086914 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.086897 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fltwp" Jul 09 04:32:10.088551 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.088534 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.089144 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.089128 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 09 04:32:10.089220 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.089198 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 09 04:32:10.089280 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.089269 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-j7r9m\"" Jul 09 04:32:10.090802 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.090782 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-g988b\"" Jul 09 04:32:10.091049 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.091033 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 09 04:32:10.091161 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.091145 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 09 04:32:10.097837 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.097812 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cz4z2\" (UniqueName: \"kubernetes.io/projected/7b2795d7-ba4b-46ea-bd2c-6b9d061e9746-kube-api-access-cz4z2\") pod \"node-resolver-xzgbm\" (UID: \"7b2795d7-ba4b-46ea-bd2c-6b9d061e9746\") " pod="openshift-dns/node-resolver-xzgbm" Jul 09 04:32:10.097925 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.097851 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ea2c59ff-64a0-4a8b-b179-6dfbf26be5df-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4g7j6\" (UID: \"ea2c59ff-64a0-4a8b-b179-6dfbf26be5df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g7j6" Jul 09 04:32:10.097925 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.097876 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ea2c59ff-64a0-4a8b-b179-6dfbf26be5df-socket-dir\") pod \"aws-ebs-csi-driver-node-4g7j6\" (UID: \"ea2c59ff-64a0-4a8b-b179-6dfbf26be5df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g7j6" Jul 09 04:32:10.097925 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.097884 2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 04:32:10.097925 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.097905 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ea2c59ff-64a0-4a8b-b179-6dfbf26be5df-registration-dir\") pod \"aws-ebs-csi-driver-node-4g7j6\" (UID: \"ea2c59ff-64a0-4a8b-b179-6dfbf26be5df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g7j6" Jul 09 04:32:10.098079 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.097934 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ea2c59ff-64a0-4a8b-b179-6dfbf26be5df-sys-fs\") pod \"aws-ebs-csi-driver-node-4g7j6\" (UID: \"ea2c59ff-64a0-4a8b-b179-6dfbf26be5df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g7j6" Jul 09 04:32:10.098079 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.097956 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/22077c07-57a6-4027-acae-197295ac8f18-iptables-alerter-script\") pod \"iptables-alerter-bzk6k\" (UID: \"22077c07-57a6-4027-acae-197295ac8f18\") " pod="openshift-network-operator/iptables-alerter-bzk6k" Jul 09 04:32:10.098079 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.097981 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-cnibin\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.098079 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098004 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b6b791a3-d352-45b2-8394-6dae85f91c7f-etc-sysctl-conf\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.098079 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098025 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7b2795d7-ba4b-46ea-bd2c-6b9d061e9746-hosts-file\") pod \"node-resolver-xzgbm\" (UID: \"7b2795d7-ba4b-46ea-bd2c-6b9d061e9746\") " pod="openshift-dns/node-resolver-xzgbm" Jul 09 04:32:10.098079 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098047 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-host-run-ovn-kubernetes\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.098079 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098068 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-host-cni-netd\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.098315 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098091 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55njz\" (UniqueName: \"kubernetes.io/projected/af45d649-376b-4efd-ad4c-9b5b3c28ab4d-kube-api-access-55njz\") pod \"multus-additional-cni-plugins-cmpgg\" (UID: \"af45d649-376b-4efd-ad4c-9b5b3c28ab4d\") " pod="openshift-multus/multus-additional-cni-plugins-cmpgg" Jul 09 04:32:10.098315 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098117 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7smhn\" (UniqueName: \"kubernetes.io/projected/22077c07-57a6-4027-acae-197295ac8f18-kube-api-access-7smhn\") pod \"iptables-alerter-bzk6k\" (UID: \"22077c07-57a6-4027-acae-197295ac8f18\") " pod="openshift-network-operator/iptables-alerter-bzk6k" Jul 09 04:32:10.098315 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098139 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-host-run-k8s-cni-cncf-io\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.098315 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098181 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd76g\" (UniqueName: \"kubernetes.io/projected/b6b791a3-d352-45b2-8394-6dae85f91c7f-kube-api-access-dd76g\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.098315 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098211 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-host-slash\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.098315 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098236 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-run-systemd\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.098315 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098257 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-node-log\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.098315 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098280 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjjnk\" (UniqueName: \"kubernetes.io/projected/227fe7b3-97ee-435b-9e07-15cf67dd2beb-kube-api-access-xjjnk\") pod \"node-ca-j7zhv\" (UID: \"227fe7b3-97ee-435b-9e07-15cf67dd2beb\") " pod="openshift-image-registry/node-ca-j7zhv" Jul 09 04:32:10.098315 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098304 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgk4z\" (UniqueName: \"kubernetes.io/projected/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-kube-api-access-mgk4z\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.098672 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098326 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/97d22a8e-85f8-4d3a-8fbf-cdff7581f85e-agent-certs\") pod \"konnectivity-agent-fltwp\" (UID: \"97d22a8e-85f8-4d3a-8fbf-cdff7581f85e\") " pod="kube-system/konnectivity-agent-fltwp" Jul 09 04:32:10.098672 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098370 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/97d22a8e-85f8-4d3a-8fbf-cdff7581f85e-konnectivity-ca\") pod \"konnectivity-agent-fltwp\" (UID: \"97d22a8e-85f8-4d3a-8fbf-cdff7581f85e\") " pod="kube-system/konnectivity-agent-fltwp" Jul 09 04:32:10.098672 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098396 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b6b791a3-d352-45b2-8394-6dae85f91c7f-lib-modules\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.098672 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098426 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/af45d649-376b-4efd-ad4c-9b5b3c28ab4d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cmpgg\" (UID: \"af45d649-376b-4efd-ad4c-9b5b3c28ab4d\") " pod="openshift-multus/multus-additional-cni-plugins-cmpgg" Jul 09 04:32:10.098672 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098452 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9k2gw\" (UniqueName: \"kubernetes.io/projected/dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa-kube-api-access-9k2gw\") pod \"network-metrics-daemon-gpx47\" (UID: \"dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa\") " pod="openshift-multus/network-metrics-daemon-gpx47" Jul 09 04:32:10.098672 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098484 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-886q7\" (UniqueName: \"kubernetes.io/projected/f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59-kube-api-access-886q7\") pod \"network-check-target-bgsl7\" (UID: \"f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59\") " pod="openshift-network-diagnostics/network-check-target-bgsl7" Jul 09 04:32:10.098672 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098509 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b6b791a3-d352-45b2-8394-6dae85f91c7f-etc-modprobe-d\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.098672 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098524 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-host-kubelet\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.098672 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098558 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/227fe7b3-97ee-435b-9e07-15cf67dd2beb-serviceca\") pod \"node-ca-j7zhv\" (UID: \"227fe7b3-97ee-435b-9e07-15cf67dd2beb\") " pod="openshift-image-registry/node-ca-j7zhv" Jul 09 04:32:10.098672 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098582 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-os-release\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.098672 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098601 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa-metrics-certs\") pod \"network-metrics-daemon-gpx47\" (UID: \"dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa\") " pod="openshift-multus/network-metrics-daemon-gpx47" Jul 09 04:32:10.098672 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098614 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b6b791a3-d352-45b2-8394-6dae85f91c7f-host\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.098672 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098634 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.098672 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098660 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/52c0430b-efce-47bf-9a4f-2428de23d349-ovn-node-metrics-cert\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.098672 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098675 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/22077c07-57a6-4027-acae-197295ac8f18-host-slash\") pod \"iptables-alerter-bzk6k\" (UID: \"22077c07-57a6-4027-acae-197295ac8f18\") " pod="openshift-network-operator/iptables-alerter-bzk6k" Jul 09 04:32:10.099241 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098689 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-multus-daemon-config\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.099241 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098711 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-host-run-multus-certs\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.099241 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098734 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-run-openvswitch\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.099241 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098753 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/af45d649-376b-4efd-ad4c-9b5b3c28ab4d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cmpgg\" (UID: \"af45d649-376b-4efd-ad4c-9b5b3c28ab4d\") " pod="openshift-multus/multus-additional-cni-plugins-cmpgg" Jul 09 04:32:10.099241 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098767 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-multus-cni-dir\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.099241 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098785 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-cni-binary-copy\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.099241 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098809 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-multus-conf-dir\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.099241 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098830 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-etc-kubernetes\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.099241 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098843 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b6b791a3-d352-45b2-8394-6dae85f91c7f-etc-sysconfig\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.099241 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098861 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6b791a3-d352-45b2-8394-6dae85f91c7f-tmp\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.099241 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098885 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/af45d649-376b-4efd-ad4c-9b5b3c28ab4d-system-cni-dir\") pod \"multus-additional-cni-plugins-cmpgg\" (UID: \"af45d649-376b-4efd-ad4c-9b5b3c28ab4d\") " pod="openshift-multus/multus-additional-cni-plugins-cmpgg" Jul 09 04:32:10.099241 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098901 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/af45d649-376b-4efd-ad4c-9b5b3c28ab4d-cni-binary-copy\") pod \"multus-additional-cni-plugins-cmpgg\" (UID: \"af45d649-376b-4efd-ad4c-9b5b3c28ab4d\") " pod="openshift-multus/multus-additional-cni-plugins-cmpgg" Jul 09 04:32:10.099241 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098924 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsjp4\" (UniqueName: \"kubernetes.io/projected/ea2c59ff-64a0-4a8b-b179-6dfbf26be5df-kube-api-access-vsjp4\") pod \"aws-ebs-csi-driver-node-4g7j6\" (UID: \"ea2c59ff-64a0-4a8b-b179-6dfbf26be5df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g7j6" Jul 09 04:32:10.099241 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098947 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b6b791a3-d352-45b2-8394-6dae85f91c7f-etc-sysctl-d\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.099241 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098969 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b6b791a3-d352-45b2-8394-6dae85f91c7f-var-lib-kubelet\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.099241 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.098991 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-hostroot\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.099925 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.099034 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6b791a3-d352-45b2-8394-6dae85f91c7f-etc-kubernetes\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.099925 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.099057 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b2795d7-ba4b-46ea-bd2c-6b9d061e9746-tmp-dir\") pod \"node-resolver-xzgbm\" (UID: \"7b2795d7-ba4b-46ea-bd2c-6b9d061e9746\") " pod="openshift-dns/node-resolver-xzgbm" Jul 09 04:32:10.099925 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.099087 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-etc-openvswitch\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.099925 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.099137 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-run-ovn\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.099925 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.099188 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/af45d649-376b-4efd-ad4c-9b5b3c28ab4d-os-release\") pod \"multus-additional-cni-plugins-cmpgg\" (UID: \"af45d649-376b-4efd-ad4c-9b5b3c28ab4d\") " pod="openshift-multus/multus-additional-cni-plugins-cmpgg" Jul 09 04:32:10.099925 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.099213 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/227fe7b3-97ee-435b-9e07-15cf67dd2beb-host\") pod \"node-ca-j7zhv\" (UID: \"227fe7b3-97ee-435b-9e07-15cf67dd2beb\") " pod="openshift-image-registry/node-ca-j7zhv" Jul 09 04:32:10.099925 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.099235 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-system-cni-dir\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.099925 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.099257 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-multus-socket-dir-parent\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.099925 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.099280 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/52c0430b-efce-47bf-9a4f-2428de23d349-env-overrides\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.099925 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.099302 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ea2c59ff-64a0-4a8b-b179-6dfbf26be5df-device-dir\") pod \"aws-ebs-csi-driver-node-4g7j6\" (UID: \"ea2c59ff-64a0-4a8b-b179-6dfbf26be5df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g7j6" Jul 09 04:32:10.099925 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.099325 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-host-var-lib-cni-multus\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.099925 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.099382 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-host-var-lib-kubelet\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.099925 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.099409 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0a12812-bbc1-4b88-a2f2-858d22d67e64-original-pull-secret\") pod \"global-pull-secret-syncer-zl4tp\" (UID: \"d0a12812-bbc1-4b88-a2f2-858d22d67e64\") " pod="kube-system/global-pull-secret-syncer-zl4tp" Jul 09 04:32:10.099925 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.099435 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/af45d649-376b-4efd-ad4c-9b5b3c28ab4d-cnibin\") pod \"multus-additional-cni-plugins-cmpgg\" (UID: \"af45d649-376b-4efd-ad4c-9b5b3c28ab4d\") " pod="openshift-multus/multus-additional-cni-plugins-cmpgg" Jul 09 04:32:10.099925 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.099457 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ea2c59ff-64a0-4a8b-b179-6dfbf26be5df-etc-selinux\") pod \"aws-ebs-csi-driver-node-4g7j6\" (UID: \"ea2c59ff-64a0-4a8b-b179-6dfbf26be5df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g7j6" Jul 09 04:32:10.099925 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.099479 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b6b791a3-d352-45b2-8394-6dae85f91c7f-etc-systemd\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.100538 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.099502 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/52c0430b-efce-47bf-9a4f-2428de23d349-ovnkube-script-lib\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.100538 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.099526 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/af45d649-376b-4efd-ad4c-9b5b3c28ab4d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cmpgg\" (UID: \"af45d649-376b-4efd-ad4c-9b5b3c28ab4d\") " pod="openshift-multus/multus-additional-cni-plugins-cmpgg" Jul 09 04:32:10.100538 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.099549 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d0a12812-bbc1-4b88-a2f2-858d22d67e64-kubelet-config\") pod \"global-pull-secret-syncer-zl4tp\" (UID: \"d0a12812-bbc1-4b88-a2f2-858d22d67e64\") " pod="kube-system/global-pull-secret-syncer-zl4tp" Jul 09 04:32:10.100538 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.099587 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-var-lib-openvswitch\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.100538 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.099618 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b6b791a3-d352-45b2-8394-6dae85f91c7f-run\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.100538 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.099642 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-systemd-units\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.100538 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.099672 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-host-run-netns\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.100538 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.099697 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtfbg\" (UniqueName: \"kubernetes.io/projected/52c0430b-efce-47bf-9a4f-2428de23d349-kube-api-access-xtfbg\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.100538 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.099720 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-host-run-netns\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.100538 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.099742 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-log-socket\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.100538 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.099765 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-host-cni-bin\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.100538 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.099788 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6b791a3-d352-45b2-8394-6dae85f91c7f-sys\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.100538 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.099810 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b6b791a3-d352-45b2-8394-6dae85f91c7f-etc-tuned\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.100538 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.099830 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/52c0430b-efce-47bf-9a4f-2428de23d349-ovnkube-config\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.100538 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.099850 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d0a12812-bbc1-4b88-a2f2-858d22d67e64-dbus\") pod \"global-pull-secret-syncer-zl4tp\" (UID: \"d0a12812-bbc1-4b88-a2f2-858d22d67e64\") " pod="kube-system/global-pull-secret-syncer-zl4tp" Jul 09 04:32:10.100538 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.099878 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-host-var-lib-cni-bin\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.120072 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.120048 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-j8pr2" Jul 09 04:32:10.126784 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.126756 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-j8pr2" Jul 09 04:32:10.187651 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.187621 2565 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 09 04:32:10.194154 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.194103 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-184.ec2.internal" event={"ID":"4e5b88752c85b426292efed1a2dab6a2","Type":"ContainerStarted","Data":"055c57fd884490d5cf11890184c54fca8b8aaa43f91c9bf6a1171a8ad4367850"} Jul 09 04:32:10.195351 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.195311 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-184.ec2.internal" event={"ID":"be23e120c66897ba7a1764c4c10c1b91","Type":"ContainerStarted","Data":"317a66ea17d1b245a4df2168e0eee51ee92dcbf8b2f901c2c6fbf29cc3785dc9"} Jul 09 04:32:10.200555 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.200531 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-multus-conf-dir\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.200683 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.200562 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-etc-kubernetes\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.200683 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.200583 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b6b791a3-d352-45b2-8394-6dae85f91c7f-etc-sysconfig\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.200683 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.200607 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6b791a3-d352-45b2-8394-6dae85f91c7f-tmp\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.200683 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.200651 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-multus-conf-dir\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.200683 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.200673 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-etc-kubernetes\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.200928 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.200715 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b6b791a3-d352-45b2-8394-6dae85f91c7f-etc-sysconfig\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.200928 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.200721 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/af45d649-376b-4efd-ad4c-9b5b3c28ab4d-system-cni-dir\") pod \"multus-additional-cni-plugins-cmpgg\" (UID: \"af45d649-376b-4efd-ad4c-9b5b3c28ab4d\") " pod="openshift-multus/multus-additional-cni-plugins-cmpgg" Jul 09 04:32:10.200928 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.200751 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/af45d649-376b-4efd-ad4c-9b5b3c28ab4d-cni-binary-copy\") pod \"multus-additional-cni-plugins-cmpgg\" (UID: \"af45d649-376b-4efd-ad4c-9b5b3c28ab4d\") " pod="openshift-multus/multus-additional-cni-plugins-cmpgg" Jul 09 04:32:10.200928 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.200756 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/af45d649-376b-4efd-ad4c-9b5b3c28ab4d-system-cni-dir\") pod \"multus-additional-cni-plugins-cmpgg\" (UID: \"af45d649-376b-4efd-ad4c-9b5b3c28ab4d\") " pod="openshift-multus/multus-additional-cni-plugins-cmpgg" Jul 09 04:32:10.200928 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.200777 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsjp4\" (UniqueName: \"kubernetes.io/projected/ea2c59ff-64a0-4a8b-b179-6dfbf26be5df-kube-api-access-vsjp4\") pod \"aws-ebs-csi-driver-node-4g7j6\" (UID: \"ea2c59ff-64a0-4a8b-b179-6dfbf26be5df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g7j6" Jul 09 04:32:10.200928 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.200804 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b6b791a3-d352-45b2-8394-6dae85f91c7f-etc-sysctl-d\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.200928 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.200834 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b6b791a3-d352-45b2-8394-6dae85f91c7f-var-lib-kubelet\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.200928 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.200860 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-hostroot\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.200928 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.200883 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6b791a3-d352-45b2-8394-6dae85f91c7f-etc-kubernetes\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.200928 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.200928 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b2795d7-ba4b-46ea-bd2c-6b9d061e9746-tmp-dir\") pod \"node-resolver-xzgbm\" (UID: \"7b2795d7-ba4b-46ea-bd2c-6b9d061e9746\") " pod="openshift-dns/node-resolver-xzgbm" Jul 09 04:32:10.201421 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.200945 2565 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 09 04:32:10.201421 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.200987 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-etc-openvswitch\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.201421 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201026 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b6b791a3-d352-45b2-8394-6dae85f91c7f-var-lib-kubelet\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.201421 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201036 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b6b791a3-d352-45b2-8394-6dae85f91c7f-etc-sysctl-d\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.201421 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201049 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-hostroot\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.201421 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201072 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6b791a3-d352-45b2-8394-6dae85f91c7f-etc-kubernetes\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.201421 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.200953 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-etc-openvswitch\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.201421 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201325 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-run-ovn\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.201421 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201327 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/af45d649-376b-4efd-ad4c-9b5b3c28ab4d-cni-binary-copy\") pod \"multus-additional-cni-plugins-cmpgg\" (UID: \"af45d649-376b-4efd-ad4c-9b5b3c28ab4d\") " pod="openshift-multus/multus-additional-cni-plugins-cmpgg" Jul 09 04:32:10.201421 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201325 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b2795d7-ba4b-46ea-bd2c-6b9d061e9746-tmp-dir\") pod \"node-resolver-xzgbm\" (UID: \"7b2795d7-ba4b-46ea-bd2c-6b9d061e9746\") " pod="openshift-dns/node-resolver-xzgbm" Jul 09 04:32:10.201421 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201376 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/af45d649-376b-4efd-ad4c-9b5b3c28ab4d-os-release\") pod \"multus-additional-cni-plugins-cmpgg\" (UID: \"af45d649-376b-4efd-ad4c-9b5b3c28ab4d\") " pod="openshift-multus/multus-additional-cni-plugins-cmpgg" Jul 09 04:32:10.201421 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201404 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/227fe7b3-97ee-435b-9e07-15cf67dd2beb-host\") pod \"node-ca-j7zhv\" (UID: \"227fe7b3-97ee-435b-9e07-15cf67dd2beb\") " pod="openshift-image-registry/node-ca-j7zhv" Jul 09 04:32:10.201421 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201428 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-run-ovn\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.202030 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201427 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-system-cni-dir\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.202030 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201460 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-multus-socket-dir-parent\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.202030 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201476 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/227fe7b3-97ee-435b-9e07-15cf67dd2beb-host\") pod \"node-ca-j7zhv\" (UID: \"227fe7b3-97ee-435b-9e07-15cf67dd2beb\") " pod="openshift-image-registry/node-ca-j7zhv" Jul 09 04:32:10.202030 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201482 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/52c0430b-efce-47bf-9a4f-2428de23d349-env-overrides\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.202030 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201510 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ea2c59ff-64a0-4a8b-b179-6dfbf26be5df-device-dir\") pod \"aws-ebs-csi-driver-node-4g7j6\" (UID: \"ea2c59ff-64a0-4a8b-b179-6dfbf26be5df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g7j6" Jul 09 04:32:10.202030 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201512 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/af45d649-376b-4efd-ad4c-9b5b3c28ab4d-os-release\") pod \"multus-additional-cni-plugins-cmpgg\" (UID: \"af45d649-376b-4efd-ad4c-9b5b3c28ab4d\") " pod="openshift-multus/multus-additional-cni-plugins-cmpgg" Jul 09 04:32:10.202030 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201533 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-host-var-lib-cni-multus\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.202030 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201555 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-host-var-lib-kubelet\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.202030 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201569 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-system-cni-dir\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.202030 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201579 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0a12812-bbc1-4b88-a2f2-858d22d67e64-original-pull-secret\") pod \"global-pull-secret-syncer-zl4tp\" (UID: \"d0a12812-bbc1-4b88-a2f2-858d22d67e64\") " pod="kube-system/global-pull-secret-syncer-zl4tp" Jul 09 04:32:10.202030 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201587 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-multus-socket-dir-parent\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.202030 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201603 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/af45d649-376b-4efd-ad4c-9b5b3c28ab4d-cnibin\") pod \"multus-additional-cni-plugins-cmpgg\" (UID: \"af45d649-376b-4efd-ad4c-9b5b3c28ab4d\") " pod="openshift-multus/multus-additional-cni-plugins-cmpgg" Jul 09 04:32:10.202030 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201628 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ea2c59ff-64a0-4a8b-b179-6dfbf26be5df-etc-selinux\") pod \"aws-ebs-csi-driver-node-4g7j6\" (UID: \"ea2c59ff-64a0-4a8b-b179-6dfbf26be5df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g7j6" Jul 09 04:32:10.202030 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201654 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b6b791a3-d352-45b2-8394-6dae85f91c7f-etc-systemd\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.202030 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201670 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-host-var-lib-kubelet\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.202030 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:10.201679 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 04:32:10.202030 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201704 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/52c0430b-efce-47bf-9a4f-2428de23d349-ovnkube-script-lib\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.202030 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201722 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/af45d649-376b-4efd-ad4c-9b5b3c28ab4d-cnibin\") pod \"multus-additional-cni-plugins-cmpgg\" (UID: \"af45d649-376b-4efd-ad4c-9b5b3c28ab4d\") " pod="openshift-multus/multus-additional-cni-plugins-cmpgg" Jul 09 04:32:10.202912 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201630 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ea2c59ff-64a0-4a8b-b179-6dfbf26be5df-device-dir\") pod \"aws-ebs-csi-driver-node-4g7j6\" (UID: \"ea2c59ff-64a0-4a8b-b179-6dfbf26be5df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g7j6" Jul 09 04:32:10.202912 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:10.201742 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0a12812-bbc1-4b88-a2f2-858d22d67e64-original-pull-secret podName:d0a12812-bbc1-4b88-a2f2-858d22d67e64 nodeName:}" failed. No retries permitted until 2026-07-09 04:32:10.701722148 +0000 UTC m=+2.953245720 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d0a12812-bbc1-4b88-a2f2-858d22d67e64-original-pull-secret") pod "global-pull-secret-syncer-zl4tp" (UID: "d0a12812-bbc1-4b88-a2f2-858d22d67e64") : object "kube-system"/"original-pull-secret" not registered Jul 09 04:32:10.202912 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201760 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/af45d649-376b-4efd-ad4c-9b5b3c28ab4d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cmpgg\" (UID: \"af45d649-376b-4efd-ad4c-9b5b3c28ab4d\") " pod="openshift-multus/multus-additional-cni-plugins-cmpgg" Jul 09 04:32:10.202912 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201779 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b6b791a3-d352-45b2-8394-6dae85f91c7f-etc-systemd\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.202912 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201788 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d0a12812-bbc1-4b88-a2f2-858d22d67e64-kubelet-config\") pod \"global-pull-secret-syncer-zl4tp\" (UID: \"d0a12812-bbc1-4b88-a2f2-858d22d67e64\") " pod="kube-system/global-pull-secret-syncer-zl4tp" Jul 09 04:32:10.202912 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201814 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-var-lib-openvswitch\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.202912 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201838 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b6b791a3-d352-45b2-8394-6dae85f91c7f-run\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.202912 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201843 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-host-var-lib-cni-multus\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.202912 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201860 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-systemd-units\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.202912 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201883 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-host-run-netns\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.202912 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201895 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d0a12812-bbc1-4b88-a2f2-858d22d67e64-kubelet-config\") pod \"global-pull-secret-syncer-zl4tp\" (UID: \"d0a12812-bbc1-4b88-a2f2-858d22d67e64\") " pod="kube-system/global-pull-secret-syncer-zl4tp" Jul 09 04:32:10.202912 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201908 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtfbg\" (UniqueName: \"kubernetes.io/projected/52c0430b-efce-47bf-9a4f-2428de23d349-kube-api-access-xtfbg\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.202912 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201932 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-host-run-netns\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.202912 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201948 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b6b791a3-d352-45b2-8394-6dae85f91c7f-run\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.202912 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201954 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-log-socket\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.202912 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201814 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ea2c59ff-64a0-4a8b-b179-6dfbf26be5df-etc-selinux\") pod \"aws-ebs-csi-driver-node-4g7j6\" (UID: \"ea2c59ff-64a0-4a8b-b179-6dfbf26be5df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g7j6" Jul 09 04:32:10.204680 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201988 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-var-lib-openvswitch\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.204680 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201979 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-host-cni-bin\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.204680 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202011 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-host-cni-bin\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.204680 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202040 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6b791a3-d352-45b2-8394-6dae85f91c7f-sys\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.204680 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202050 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-systemd-units\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.204680 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.201860 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/52c0430b-efce-47bf-9a4f-2428de23d349-env-overrides\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.204680 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202072 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b6b791a3-d352-45b2-8394-6dae85f91c7f-etc-tuned\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.204680 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202098 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/52c0430b-efce-47bf-9a4f-2428de23d349-ovnkube-config\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.204680 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202124 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d0a12812-bbc1-4b88-a2f2-858d22d67e64-dbus\") pod \"global-pull-secret-syncer-zl4tp\" (UID: \"d0a12812-bbc1-4b88-a2f2-858d22d67e64\") " pod="kube-system/global-pull-secret-syncer-zl4tp" Jul 09 04:32:10.204680 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202136 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-host-run-netns\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.204680 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202149 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-host-var-lib-cni-bin\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.204680 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202174 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cz4z2\" (UniqueName: \"kubernetes.io/projected/7b2795d7-ba4b-46ea-bd2c-6b9d061e9746-kube-api-access-cz4z2\") pod \"node-resolver-xzgbm\" (UID: \"7b2795d7-ba4b-46ea-bd2c-6b9d061e9746\") " pod="openshift-dns/node-resolver-xzgbm" Jul 09 04:32:10.204680 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202100 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-host-run-netns\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.204680 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202200 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ea2c59ff-64a0-4a8b-b179-6dfbf26be5df-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4g7j6\" (UID: \"ea2c59ff-64a0-4a8b-b179-6dfbf26be5df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g7j6" Jul 09 04:32:10.204680 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202224 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ea2c59ff-64a0-4a8b-b179-6dfbf26be5df-socket-dir\") pod \"aws-ebs-csi-driver-node-4g7j6\" (UID: \"ea2c59ff-64a0-4a8b-b179-6dfbf26be5df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g7j6" Jul 09 04:32:10.204680 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202234 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/af45d649-376b-4efd-ad4c-9b5b3c28ab4d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cmpgg\" (UID: \"af45d649-376b-4efd-ad4c-9b5b3c28ab4d\") " pod="openshift-multus/multus-additional-cni-plugins-cmpgg" Jul 09 04:32:10.204680 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202238 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6b791a3-d352-45b2-8394-6dae85f91c7f-sys\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.205532 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202247 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ea2c59ff-64a0-4a8b-b179-6dfbf26be5df-registration-dir\") pod \"aws-ebs-csi-driver-node-4g7j6\" (UID: \"ea2c59ff-64a0-4a8b-b179-6dfbf26be5df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g7j6" Jul 09 04:32:10.205532 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202259 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/52c0430b-efce-47bf-9a4f-2428de23d349-ovnkube-script-lib\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.205532 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202289 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-host-var-lib-cni-bin\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.205532 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202369 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ea2c59ff-64a0-4a8b-b179-6dfbf26be5df-socket-dir\") pod \"aws-ebs-csi-driver-node-4g7j6\" (UID: \"ea2c59ff-64a0-4a8b-b179-6dfbf26be5df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g7j6" Jul 09 04:32:10.205532 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202184 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-log-socket\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.205532 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202403 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d0a12812-bbc1-4b88-a2f2-858d22d67e64-dbus\") pod \"global-pull-secret-syncer-zl4tp\" (UID: \"d0a12812-bbc1-4b88-a2f2-858d22d67e64\") " pod="kube-system/global-pull-secret-syncer-zl4tp" Jul 09 04:32:10.205532 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202418 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ea2c59ff-64a0-4a8b-b179-6dfbf26be5df-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4g7j6\" (UID: \"ea2c59ff-64a0-4a8b-b179-6dfbf26be5df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g7j6" Jul 09 04:32:10.205532 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202472 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ea2c59ff-64a0-4a8b-b179-6dfbf26be5df-sys-fs\") pod \"aws-ebs-csi-driver-node-4g7j6\" (UID: \"ea2c59ff-64a0-4a8b-b179-6dfbf26be5df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g7j6" Jul 09 04:32:10.205532 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202501 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/22077c07-57a6-4027-acae-197295ac8f18-iptables-alerter-script\") pod \"iptables-alerter-bzk6k\" (UID: \"22077c07-57a6-4027-acae-197295ac8f18\") " pod="openshift-network-operator/iptables-alerter-bzk6k" Jul 09 04:32:10.205532 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202533 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-cnibin\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.205532 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202557 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ea2c59ff-64a0-4a8b-b179-6dfbf26be5df-sys-fs\") pod \"aws-ebs-csi-driver-node-4g7j6\" (UID: \"ea2c59ff-64a0-4a8b-b179-6dfbf26be5df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g7j6" Jul 09 04:32:10.205532 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202558 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b6b791a3-d352-45b2-8394-6dae85f91c7f-etc-sysctl-conf\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.205532 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202594 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7b2795d7-ba4b-46ea-bd2c-6b9d061e9746-hosts-file\") pod \"node-resolver-xzgbm\" (UID: \"7b2795d7-ba4b-46ea-bd2c-6b9d061e9746\") " pod="openshift-dns/node-resolver-xzgbm" Jul 09 04:32:10.205532 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202620 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-host-run-ovn-kubernetes\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.205532 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202648 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-host-cni-netd\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.205532 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202658 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b6b791a3-d352-45b2-8394-6dae85f91c7f-etc-sysctl-conf\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.205532 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202670 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7b2795d7-ba4b-46ea-bd2c-6b9d061e9746-hosts-file\") pod \"node-resolver-xzgbm\" (UID: \"7b2795d7-ba4b-46ea-bd2c-6b9d061e9746\") " pod="openshift-dns/node-resolver-xzgbm" Jul 09 04:32:10.206314 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202676 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55njz\" (UniqueName: \"kubernetes.io/projected/af45d649-376b-4efd-ad4c-9b5b3c28ab4d-kube-api-access-55njz\") pod \"multus-additional-cni-plugins-cmpgg\" (UID: \"af45d649-376b-4efd-ad4c-9b5b3c28ab4d\") " pod="openshift-multus/multus-additional-cni-plugins-cmpgg" Jul 09 04:32:10.206314 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202689 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/52c0430b-efce-47bf-9a4f-2428de23d349-ovnkube-config\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.206314 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202703 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7smhn\" (UniqueName: \"kubernetes.io/projected/22077c07-57a6-4027-acae-197295ac8f18-kube-api-access-7smhn\") pod \"iptables-alerter-bzk6k\" (UID: \"22077c07-57a6-4027-acae-197295ac8f18\") " pod="openshift-network-operator/iptables-alerter-bzk6k" Jul 09 04:32:10.206314 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202727 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-host-run-ovn-kubernetes\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.206314 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202733 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-host-run-k8s-cni-cncf-io\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.206314 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202740 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ea2c59ff-64a0-4a8b-b179-6dfbf26be5df-registration-dir\") pod \"aws-ebs-csi-driver-node-4g7j6\" (UID: \"ea2c59ff-64a0-4a8b-b179-6dfbf26be5df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g7j6" Jul 09 04:32:10.206314 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202768 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd76g\" (UniqueName: \"kubernetes.io/projected/b6b791a3-d352-45b2-8394-6dae85f91c7f-kube-api-access-dd76g\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.206314 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202775 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-host-run-k8s-cni-cncf-io\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.206314 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202789 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-cnibin\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.206314 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202824 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-host-cni-netd\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.206314 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.202967 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-host-slash\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.206314 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203008 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-run-systemd\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.206314 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203033 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-node-log\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.206314 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203058 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjjnk\" (UniqueName: \"kubernetes.io/projected/227fe7b3-97ee-435b-9e07-15cf67dd2beb-kube-api-access-xjjnk\") pod \"node-ca-j7zhv\" (UID: \"227fe7b3-97ee-435b-9e07-15cf67dd2beb\") " pod="openshift-image-registry/node-ca-j7zhv" Jul 09 04:32:10.206314 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203083 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgk4z\" (UniqueName: \"kubernetes.io/projected/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-kube-api-access-mgk4z\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.206314 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203093 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-run-systemd\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.206314 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203106 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/97d22a8e-85f8-4d3a-8fbf-cdff7581f85e-agent-certs\") pod \"konnectivity-agent-fltwp\" (UID: \"97d22a8e-85f8-4d3a-8fbf-cdff7581f85e\") " pod="kube-system/konnectivity-agent-fltwp" Jul 09 04:32:10.206989 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203120 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-node-log\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.206989 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203143 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-host-slash\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.206989 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203139 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/97d22a8e-85f8-4d3a-8fbf-cdff7581f85e-konnectivity-ca\") pod \"konnectivity-agent-fltwp\" (UID: \"97d22a8e-85f8-4d3a-8fbf-cdff7581f85e\") " pod="kube-system/konnectivity-agent-fltwp" Jul 09 04:32:10.206989 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203176 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b6b791a3-d352-45b2-8394-6dae85f91c7f-lib-modules\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.206989 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203183 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/22077c07-57a6-4027-acae-197295ac8f18-iptables-alerter-script\") pod \"iptables-alerter-bzk6k\" (UID: \"22077c07-57a6-4027-acae-197295ac8f18\") " pod="openshift-network-operator/iptables-alerter-bzk6k" Jul 09 04:32:10.206989 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203198 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/af45d649-376b-4efd-ad4c-9b5b3c28ab4d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cmpgg\" (UID: \"af45d649-376b-4efd-ad4c-9b5b3c28ab4d\") " pod="openshift-multus/multus-additional-cni-plugins-cmpgg" Jul 09 04:32:10.206989 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203230 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9k2gw\" (UniqueName: \"kubernetes.io/projected/dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa-kube-api-access-9k2gw\") pod \"network-metrics-daemon-gpx47\" (UID: \"dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa\") " pod="openshift-multus/network-metrics-daemon-gpx47" Jul 09 04:32:10.206989 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203254 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-886q7\" (UniqueName: \"kubernetes.io/projected/f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59-kube-api-access-886q7\") pod \"network-check-target-bgsl7\" (UID: \"f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59\") " pod="openshift-network-diagnostics/network-check-target-bgsl7" Jul 09 04:32:10.206989 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203281 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b6b791a3-d352-45b2-8394-6dae85f91c7f-etc-modprobe-d\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.206989 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203303 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-host-kubelet\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.206989 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203325 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/227fe7b3-97ee-435b-9e07-15cf67dd2beb-serviceca\") pod \"node-ca-j7zhv\" (UID: \"227fe7b3-97ee-435b-9e07-15cf67dd2beb\") " pod="openshift-image-registry/node-ca-j7zhv" Jul 09 04:32:10.206989 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203364 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-os-release\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.206989 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203436 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-os-release\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.206989 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203442 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b6b791a3-d352-45b2-8394-6dae85f91c7f-etc-modprobe-d\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.206989 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203441 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b6b791a3-d352-45b2-8394-6dae85f91c7f-lib-modules\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.206989 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203490 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-host-kubelet\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.206989 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203535 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa-metrics-certs\") pod \"network-metrics-daemon-gpx47\" (UID: \"dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa\") " pod="openshift-multus/network-metrics-daemon-gpx47" Jul 09 04:32:10.207650 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203649 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b6b791a3-d352-45b2-8394-6dae85f91c7f-host\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.207650 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203677 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.207650 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203705 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/52c0430b-efce-47bf-9a4f-2428de23d349-ovn-node-metrics-cert\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.207650 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203764 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.207650 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203772 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/97d22a8e-85f8-4d3a-8fbf-cdff7581f85e-konnectivity-ca\") pod \"konnectivity-agent-fltwp\" (UID: \"97d22a8e-85f8-4d3a-8fbf-cdff7581f85e\") " pod="kube-system/konnectivity-agent-fltwp" Jul 09 04:32:10.207650 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203793 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/22077c07-57a6-4027-acae-197295ac8f18-host-slash\") pod \"iptables-alerter-bzk6k\" (UID: \"22077c07-57a6-4027-acae-197295ac8f18\") " pod="openshift-network-operator/iptables-alerter-bzk6k" Jul 09 04:32:10.207650 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203818 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-multus-daemon-config\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.207650 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203832 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/af45d649-376b-4efd-ad4c-9b5b3c28ab4d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cmpgg\" (UID: \"af45d649-376b-4efd-ad4c-9b5b3c28ab4d\") " pod="openshift-multus/multus-additional-cni-plugins-cmpgg" Jul 09 04:32:10.207650 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203843 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-host-run-multus-certs\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.207650 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203865 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-run-openvswitch\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.207650 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203909 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/22077c07-57a6-4027-acae-197295ac8f18-host-slash\") pod \"iptables-alerter-bzk6k\" (UID: \"22077c07-57a6-4027-acae-197295ac8f18\") " pod="openshift-network-operator/iptables-alerter-bzk6k" Jul 09 04:32:10.207650 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203925 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/227fe7b3-97ee-435b-9e07-15cf67dd2beb-serviceca\") pod \"node-ca-j7zhv\" (UID: \"227fe7b3-97ee-435b-9e07-15cf67dd2beb\") " pod="openshift-image-registry/node-ca-j7zhv" Jul 09 04:32:10.207650 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203952 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52c0430b-efce-47bf-9a4f-2428de23d349-run-openvswitch\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.207650 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203975 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b6b791a3-d352-45b2-8394-6dae85f91c7f-host\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.207650 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.203994 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/af45d649-376b-4efd-ad4c-9b5b3c28ab4d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cmpgg\" (UID: \"af45d649-376b-4efd-ad4c-9b5b3c28ab4d\") " pod="openshift-multus/multus-additional-cni-plugins-cmpgg" Jul 09 04:32:10.207650 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:10.204023 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:32:10.207650 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.204025 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-multus-cni-dir\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.207650 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.204062 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-cni-binary-copy\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.208529 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.204090 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-multus-cni-dir\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.208529 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:10.204154 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa-metrics-certs podName:dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa nodeName:}" failed. No retries permitted until 2026-07-09 04:32:10.704136624 +0000 UTC m=+2.955660180 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa-metrics-certs") pod "network-metrics-daemon-gpx47" (UID: "dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:32:10.208529 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.204324 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/af45d649-376b-4efd-ad4c-9b5b3c28ab4d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cmpgg\" (UID: \"af45d649-376b-4efd-ad4c-9b5b3c28ab4d\") " pod="openshift-multus/multus-additional-cni-plugins-cmpgg" Jul 09 04:32:10.208529 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.204379 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-host-run-multus-certs\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.208529 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.204463 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-multus-daemon-config\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.208529 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.204622 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-cni-binary-copy\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.208529 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.205583 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6b791a3-d352-45b2-8394-6dae85f91c7f-tmp\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.208529 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.206135 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/97d22a8e-85f8-4d3a-8fbf-cdff7581f85e-agent-certs\") pod \"konnectivity-agent-fltwp\" (UID: \"97d22a8e-85f8-4d3a-8fbf-cdff7581f85e\") " pod="kube-system/konnectivity-agent-fltwp" Jul 09 04:32:10.208529 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.207354 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/52c0430b-efce-47bf-9a4f-2428de23d349-ovn-node-metrics-cert\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.208529 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.207789 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b6b791a3-d352-45b2-8394-6dae85f91c7f-etc-tuned\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.213751 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.213730 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd76g\" (UniqueName: \"kubernetes.io/projected/b6b791a3-d352-45b2-8394-6dae85f91c7f-kube-api-access-dd76g\") pod \"tuned-vkz4b\" (UID: \"b6b791a3-d352-45b2-8394-6dae85f91c7f\") " pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.213847 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.213822 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtfbg\" (UniqueName: \"kubernetes.io/projected/52c0430b-efce-47bf-9a4f-2428de23d349-kube-api-access-xtfbg\") pod \"ovnkube-node-t85dg\" (UID: \"52c0430b-efce-47bf-9a4f-2428de23d349\") " pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.220108 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.220059 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsjp4\" (UniqueName: \"kubernetes.io/projected/ea2c59ff-64a0-4a8b-b179-6dfbf26be5df-kube-api-access-vsjp4\") pod \"aws-ebs-csi-driver-node-4g7j6\" (UID: \"ea2c59ff-64a0-4a8b-b179-6dfbf26be5df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g7j6" Jul 09 04:32:10.220247 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.220231 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgk4z\" (UniqueName: \"kubernetes.io/projected/2b4a7d33-f06c-489c-ab25-b7b5b109d33c-kube-api-access-mgk4z\") pod \"multus-2tgfm\" (UID: \"2b4a7d33-f06c-489c-ab25-b7b5b109d33c\") " pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.221628 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:10.221370 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 04:32:10.221628 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:10.221392 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 04:32:10.221628 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:10.221407 2565 projected.go:196] Error preparing data for projected volume kube-api-access-886q7 for pod openshift-network-diagnostics/network-check-target-bgsl7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:32:10.221628 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:10.221465 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59-kube-api-access-886q7 podName:f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59 nodeName:}" failed. No retries permitted until 2026-07-09 04:32:10.721447264 +0000 UTC m=+2.972970823 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-886q7" (UniqueName: "kubernetes.io/projected/f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59-kube-api-access-886q7") pod "network-check-target-bgsl7" (UID: "f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:32:10.222046 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.222018 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9k2gw\" (UniqueName: \"kubernetes.io/projected/dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa-kube-api-access-9k2gw\") pod \"network-metrics-daemon-gpx47\" (UID: \"dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa\") " pod="openshift-multus/network-metrics-daemon-gpx47" Jul 09 04:32:10.222452 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.222428 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cz4z2\" (UniqueName: \"kubernetes.io/projected/7b2795d7-ba4b-46ea-bd2c-6b9d061e9746-kube-api-access-cz4z2\") pod \"node-resolver-xzgbm\" (UID: \"7b2795d7-ba4b-46ea-bd2c-6b9d061e9746\") " pod="openshift-dns/node-resolver-xzgbm" Jul 09 04:32:10.223491 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.223472 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55njz\" (UniqueName: \"kubernetes.io/projected/af45d649-376b-4efd-ad4c-9b5b3c28ab4d-kube-api-access-55njz\") pod \"multus-additional-cni-plugins-cmpgg\" (UID: \"af45d649-376b-4efd-ad4c-9b5b3c28ab4d\") " pod="openshift-multus/multus-additional-cni-plugins-cmpgg" Jul 09 04:32:10.225043 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.224969 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7smhn\" (UniqueName: \"kubernetes.io/projected/22077c07-57a6-4027-acae-197295ac8f18-kube-api-access-7smhn\") pod \"iptables-alerter-bzk6k\" (UID: \"22077c07-57a6-4027-acae-197295ac8f18\") " pod="openshift-network-operator/iptables-alerter-bzk6k" Jul 09 04:32:10.228919 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.228896 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjjnk\" (UniqueName: \"kubernetes.io/projected/227fe7b3-97ee-435b-9e07-15cf67dd2beb-kube-api-access-xjjnk\") pod \"node-ca-j7zhv\" (UID: \"227fe7b3-97ee-435b-9e07-15cf67dd2beb\") " pod="openshift-image-registry/node-ca-j7zhv" Jul 09 04:32:10.386043 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.385959 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g7j6" Jul 09 04:32:10.395170 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.395145 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xzgbm" Jul 09 04:32:10.405668 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.405622 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bzk6k" Jul 09 04:32:10.413286 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.413255 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:10.418881 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.418858 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-j7zhv" Jul 09 04:32:10.426517 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.426495 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cmpgg" Jul 09 04:32:10.434004 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.433981 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2tgfm" Jul 09 04:32:10.445539 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.445510 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fltwp" Jul 09 04:32:10.449208 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.449189 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" Jul 09 04:32:10.708092 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.708060 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa-metrics-certs\") pod \"network-metrics-daemon-gpx47\" (UID: \"dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa\") " pod="openshift-multus/network-metrics-daemon-gpx47" Jul 09 04:32:10.708289 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.708113 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0a12812-bbc1-4b88-a2f2-858d22d67e64-original-pull-secret\") pod \"global-pull-secret-syncer-zl4tp\" (UID: \"d0a12812-bbc1-4b88-a2f2-858d22d67e64\") " pod="kube-system/global-pull-secret-syncer-zl4tp" Jul 09 04:32:10.708289 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:10.708208 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 04:32:10.708289 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:10.708210 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:32:10.708289 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:10.708259 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0a12812-bbc1-4b88-a2f2-858d22d67e64-original-pull-secret podName:d0a12812-bbc1-4b88-a2f2-858d22d67e64 nodeName:}" failed. No retries permitted until 2026-07-09 04:32:11.708246024 +0000 UTC m=+3.959769579 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d0a12812-bbc1-4b88-a2f2-858d22d67e64-original-pull-secret") pod "global-pull-secret-syncer-zl4tp" (UID: "d0a12812-bbc1-4b88-a2f2-858d22d67e64") : object "kube-system"/"original-pull-secret" not registered Jul 09 04:32:10.708289 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:10.708273 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa-metrics-certs podName:dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa nodeName:}" failed. No retries permitted until 2026-07-09 04:32:11.708266431 +0000 UTC m=+3.959789987 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa-metrics-certs") pod "network-metrics-daemon-gpx47" (UID: "dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:32:10.809456 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:10.809424 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-886q7\" (UniqueName: \"kubernetes.io/projected/f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59-kube-api-access-886q7\") pod \"network-check-target-bgsl7\" (UID: \"f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59\") " pod="openshift-network-diagnostics/network-check-target-bgsl7" Jul 09 04:32:10.809671 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:10.809652 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 04:32:10.809757 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:10.809677 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 04:32:10.809757 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:10.809689 2565 projected.go:196] Error preparing data for projected volume kube-api-access-886q7 for pod openshift-network-diagnostics/network-check-target-bgsl7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:32:10.809757 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:10.809741 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59-kube-api-access-886q7 podName:f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59 nodeName:}" failed. No retries permitted until 2026-07-09 04:32:11.809723381 +0000 UTC m=+4.061246940 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-886q7" (UniqueName: "kubernetes.io/projected/f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59-kube-api-access-886q7") pod "network-check-target-bgsl7" (UID: "f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:32:11.129070 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:11.128969 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 04:27:10 +0000 UTC" deadline="2027-12-12 20:01:33.523972506 +0000 UTC" Jul 09 04:32:11.129070 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:11.129007 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12519h29m22.394970738s" Jul 09 04:32:11.301852 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:11.301817 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b2795d7_ba4b_46ea_bd2c_6b9d061e9746.slice/crio-2a4079523da7f48cf855202e0570e2dfc7f40f52a5f20b30a847b2c4981d4c61 WatchSource:0}: Error finding container 2a4079523da7f48cf855202e0570e2dfc7f40f52a5f20b30a847b2c4981d4c61: Status 404 returned error can't find the container with id 2a4079523da7f48cf855202e0570e2dfc7f40f52a5f20b30a847b2c4981d4c61 Jul 09 04:32:11.310139 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:11.310105 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod227fe7b3_97ee_435b_9e07_15cf67dd2beb.slice/crio-859a6c8fd13e78e596b26e1d0af9d0e3bd9a3adc44466c5ea7da177f4ffa8804 WatchSource:0}: Error finding container 859a6c8fd13e78e596b26e1d0af9d0e3bd9a3adc44466c5ea7da177f4ffa8804: Status 404 returned error can't find the container with id 859a6c8fd13e78e596b26e1d0af9d0e3bd9a3adc44466c5ea7da177f4ffa8804 Jul 09 04:32:11.311155 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:11.311127 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22077c07_57a6_4027_acae_197295ac8f18.slice/crio-879014bf91ff710d38d073583a0aa644c9c20aee1c921653a71e8f57b21f6954 WatchSource:0}: Error finding container 879014bf91ff710d38d073583a0aa644c9c20aee1c921653a71e8f57b21f6954: Status 404 returned error can't find the container with id 879014bf91ff710d38d073583a0aa644c9c20aee1c921653a71e8f57b21f6954 Jul 09 04:32:11.311968 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:11.311947 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97d22a8e_85f8_4d3a_8fbf_cdff7581f85e.slice/crio-655fb3e015a43950539371370b7713f698c7f1674a779acf007efe501ff2b263 WatchSource:0}: Error finding container 655fb3e015a43950539371370b7713f698c7f1674a779acf007efe501ff2b263: Status 404 returned error can't find the container with id 655fb3e015a43950539371370b7713f698c7f1674a779acf007efe501ff2b263 Jul 09 04:32:11.312790 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:11.312766 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6b791a3_d352_45b2_8394_6dae85f91c7f.slice/crio-1ef9b8beaf226f8ab71ccd4e93fa593550418d3797204889811f28f16476f54b WatchSource:0}: Error finding container 1ef9b8beaf226f8ab71ccd4e93fa593550418d3797204889811f28f16476f54b: Status 404 returned error can't find the container with id 1ef9b8beaf226f8ab71ccd4e93fa593550418d3797204889811f28f16476f54b Jul 09 04:32:11.313827 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:11.313809 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52c0430b_efce_47bf_9a4f_2428de23d349.slice/crio-3086e3db730120f04dda13b0ac69c17d3ce36e0799cc1d4254803f324085cecf WatchSource:0}: Error finding container 3086e3db730120f04dda13b0ac69c17d3ce36e0799cc1d4254803f324085cecf: Status 404 returned error can't find the container with id 3086e3db730120f04dda13b0ac69c17d3ce36e0799cc1d4254803f324085cecf Jul 09 04:32:11.315625 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:11.315594 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b4a7d33_f06c_489c_ab25_b7b5b109d33c.slice/crio-445aa3accc1b83929ff03abd9685f13d2ad909d03116d2e80074e2ffaf6980b4 WatchSource:0}: Error finding container 445aa3accc1b83929ff03abd9685f13d2ad909d03116d2e80074e2ffaf6980b4: Status 404 returned error can't find the container with id 445aa3accc1b83929ff03abd9685f13d2ad909d03116d2e80074e2ffaf6980b4 Jul 09 04:32:11.316839 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:11.316821 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea2c59ff_64a0_4a8b_b179_6dfbf26be5df.slice/crio-bd09f88cc2fc09c02718976e8a57fc9a80a26ae14d1d880ef60e9da9a5082d6f WatchSource:0}: Error finding container bd09f88cc2fc09c02718976e8a57fc9a80a26ae14d1d880ef60e9da9a5082d6f: Status 404 returned error can't find the container with id bd09f88cc2fc09c02718976e8a57fc9a80a26ae14d1d880ef60e9da9a5082d6f Jul 09 04:32:11.317892 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:11.317864 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf45d649_376b_4efd_ad4c_9b5b3c28ab4d.slice/crio-cd30ed238cf53be3edec5b9a99d1ec4b3efbe21e14ce1cd01bccc171a69175e5 WatchSource:0}: Error finding container cd30ed238cf53be3edec5b9a99d1ec4b3efbe21e14ce1cd01bccc171a69175e5: Status 404 returned error can't find the container with id cd30ed238cf53be3edec5b9a99d1ec4b3efbe21e14ce1cd01bccc171a69175e5 Jul 09 04:32:11.715035 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:11.714850 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0a12812-bbc1-4b88-a2f2-858d22d67e64-original-pull-secret\") pod \"global-pull-secret-syncer-zl4tp\" (UID: \"d0a12812-bbc1-4b88-a2f2-858d22d67e64\") " pod="kube-system/global-pull-secret-syncer-zl4tp" Jul 09 04:32:11.715234 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:11.715091 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa-metrics-certs\") pod \"network-metrics-daemon-gpx47\" (UID: \"dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa\") " pod="openshift-multus/network-metrics-daemon-gpx47" Jul 09 04:32:11.715234 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:11.714994 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 04:32:11.715234 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:11.715180 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:32:11.715234 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:11.715182 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0a12812-bbc1-4b88-a2f2-858d22d67e64-original-pull-secret podName:d0a12812-bbc1-4b88-a2f2-858d22d67e64 nodeName:}" failed. No retries permitted until 2026-07-09 04:32:13.715158621 +0000 UTC m=+5.966682188 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d0a12812-bbc1-4b88-a2f2-858d22d67e64-original-pull-secret") pod "global-pull-secret-syncer-zl4tp" (UID: "d0a12812-bbc1-4b88-a2f2-858d22d67e64") : object "kube-system"/"original-pull-secret" not registered Jul 09 04:32:11.715234 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:11.715233 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa-metrics-certs podName:dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa nodeName:}" failed. No retries permitted until 2026-07-09 04:32:13.715216831 +0000 UTC m=+5.966740390 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa-metrics-certs") pod "network-metrics-daemon-gpx47" (UID: "dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:32:11.816004 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:11.815367 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-886q7\" (UniqueName: \"kubernetes.io/projected/f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59-kube-api-access-886q7\") pod \"network-check-target-bgsl7\" (UID: \"f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59\") " pod="openshift-network-diagnostics/network-check-target-bgsl7" Jul 09 04:32:11.816004 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:11.815577 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 04:32:11.816004 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:11.815598 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 04:32:11.816004 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:11.815611 2565 projected.go:196] Error preparing data for projected volume kube-api-access-886q7 for pod openshift-network-diagnostics/network-check-target-bgsl7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:32:11.816004 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:11.815665 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59-kube-api-access-886q7 podName:f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59 nodeName:}" failed. No retries permitted until 2026-07-09 04:32:13.815647261 +0000 UTC m=+6.067170832 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-886q7" (UniqueName: "kubernetes.io/projected/f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59-kube-api-access-886q7") pod "network-check-target-bgsl7" (UID: "f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:32:12.130035 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:12.129939 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 04:27:10 +0000 UTC" deadline="2027-12-07 02:41:31.079850686 +0000 UTC" Jul 09 04:32:12.130035 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:12.129983 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12382h9m18.949872291s" Jul 09 04:32:12.189820 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:12.189783 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpx47" Jul 09 04:32:12.189999 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:12.189929 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpx47" podUID="dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa" Jul 09 04:32:12.190333 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:12.190313 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bgsl7" Jul 09 04:32:12.194892 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:12.194849 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bgsl7" podUID="f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59" Jul 09 04:32:12.195035 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:12.194979 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zl4tp" Jul 09 04:32:12.195161 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:12.195117 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zl4tp" podUID="d0a12812-bbc1-4b88-a2f2-858d22d67e64" Jul 09 04:32:12.206745 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:12.206167 2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 04:32:12.210415 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:12.210332 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-j7zhv" event={"ID":"227fe7b3-97ee-435b-9e07-15cf67dd2beb","Type":"ContainerStarted","Data":"859a6c8fd13e78e596b26e1d0af9d0e3bd9a3adc44466c5ea7da177f4ffa8804"} Jul 09 04:32:12.214358 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:12.214311 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xzgbm" event={"ID":"7b2795d7-ba4b-46ea-bd2c-6b9d061e9746","Type":"ContainerStarted","Data":"2a4079523da7f48cf855202e0570e2dfc7f40f52a5f20b30a847b2c4981d4c61"} Jul 09 04:32:12.217312 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:12.217283 2565 generic.go:356] "Generic (PLEG): container finished" podID="4e5b88752c85b426292efed1a2dab6a2" containerID="0b77f4e668f3a974a8d3820a7faa3c926a36774b23308dcfc5f505a99fa61add" exitCode=0 Jul 09 04:32:12.217438 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:12.217379 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-184.ec2.internal" event={"ID":"4e5b88752c85b426292efed1a2dab6a2","Type":"ContainerDied","Data":"0b77f4e668f3a974a8d3820a7faa3c926a36774b23308dcfc5f505a99fa61add"} Jul 09 04:32:12.227705 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:12.227677 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bzk6k" event={"ID":"22077c07-57a6-4027-acae-197295ac8f18","Type":"ContainerStarted","Data":"879014bf91ff710d38d073583a0aa644c9c20aee1c921653a71e8f57b21f6954"} Jul 09 04:32:12.237139 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:12.237102 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" event={"ID":"52c0430b-efce-47bf-9a4f-2428de23d349","Type":"ContainerStarted","Data":"3086e3db730120f04dda13b0ac69c17d3ce36e0799cc1d4254803f324085cecf"} Jul 09 04:32:12.246523 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:12.246475 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fltwp" event={"ID":"97d22a8e-85f8-4d3a-8fbf-cdff7581f85e","Type":"ContainerStarted","Data":"655fb3e015a43950539371370b7713f698c7f1674a779acf007efe501ff2b263"} Jul 09 04:32:12.265121 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:12.265079 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-184.ec2.internal" event={"ID":"be23e120c66897ba7a1764c4c10c1b91","Type":"ContainerStarted","Data":"022c350516e04cbfe6ca067f409d8df1908c836a0dd6d898683a97c8384c54e2"} Jul 09 04:32:12.281146 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:12.281087 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-184.ec2.internal" podStartSLOduration=3.281066285 podStartE2EDuration="3.281066285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:32:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:32:12.280243481 +0000 UTC m=+4.531767061" watchObservedRunningTime="2026-07-09 04:32:12.281066285 +0000 UTC m=+4.532589865" Jul 09 04:32:12.286942 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:12.286908 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cmpgg" event={"ID":"af45d649-376b-4efd-ad4c-9b5b3c28ab4d","Type":"ContainerStarted","Data":"cd30ed238cf53be3edec5b9a99d1ec4b3efbe21e14ce1cd01bccc171a69175e5"} Jul 09 04:32:12.297527 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:12.297492 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g7j6" event={"ID":"ea2c59ff-64a0-4a8b-b179-6dfbf26be5df","Type":"ContainerStarted","Data":"bd09f88cc2fc09c02718976e8a57fc9a80a26ae14d1d880ef60e9da9a5082d6f"} Jul 09 04:32:12.307482 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:12.307402 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2tgfm" event={"ID":"2b4a7d33-f06c-489c-ab25-b7b5b109d33c","Type":"ContainerStarted","Data":"445aa3accc1b83929ff03abd9685f13d2ad909d03116d2e80074e2ffaf6980b4"} Jul 09 04:32:12.317592 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:12.317558 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" event={"ID":"b6b791a3-d352-45b2-8394-6dae85f91c7f","Type":"ContainerStarted","Data":"1ef9b8beaf226f8ab71ccd4e93fa593550418d3797204889811f28f16476f54b"} Jul 09 04:32:13.369634 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:13.369565 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-184.ec2.internal" event={"ID":"4e5b88752c85b426292efed1a2dab6a2","Type":"ContainerStarted","Data":"8c46c81d3b1cc8dffebe2cef97a71f07812aa2cee33fbd98cb3125b4e9766544"} Jul 09 04:32:13.736122 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:13.736083 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0a12812-bbc1-4b88-a2f2-858d22d67e64-original-pull-secret\") pod \"global-pull-secret-syncer-zl4tp\" (UID: \"d0a12812-bbc1-4b88-a2f2-858d22d67e64\") " pod="kube-system/global-pull-secret-syncer-zl4tp" Jul 09 04:32:13.736308 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:13.736178 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa-metrics-certs\") pod \"network-metrics-daemon-gpx47\" (UID: \"dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa\") " pod="openshift-multus/network-metrics-daemon-gpx47" Jul 09 04:32:13.736308 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:13.736301 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:32:13.736474 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:13.736378 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa-metrics-certs podName:dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa nodeName:}" failed. No retries permitted until 2026-07-09 04:32:17.736358249 +0000 UTC m=+9.987881808 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa-metrics-certs") pod "network-metrics-daemon-gpx47" (UID: "dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:32:13.736804 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:13.736779 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 04:32:13.736909 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:13.736841 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0a12812-bbc1-4b88-a2f2-858d22d67e64-original-pull-secret podName:d0a12812-bbc1-4b88-a2f2-858d22d67e64 nodeName:}" failed. No retries permitted until 2026-07-09 04:32:17.736823712 +0000 UTC m=+9.988347286 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d0a12812-bbc1-4b88-a2f2-858d22d67e64-original-pull-secret") pod "global-pull-secret-syncer-zl4tp" (UID: "d0a12812-bbc1-4b88-a2f2-858d22d67e64") : object "kube-system"/"original-pull-secret" not registered Jul 09 04:32:13.837012 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:13.836975 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-886q7\" (UniqueName: \"kubernetes.io/projected/f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59-kube-api-access-886q7\") pod \"network-check-target-bgsl7\" (UID: \"f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59\") " pod="openshift-network-diagnostics/network-check-target-bgsl7" Jul 09 04:32:13.837181 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:13.837133 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 04:32:13.837181 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:13.837155 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 04:32:13.837181 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:13.837168 2565 projected.go:196] Error preparing data for projected volume kube-api-access-886q7 for pod openshift-network-diagnostics/network-check-target-bgsl7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:32:13.837384 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:13.837227 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59-kube-api-access-886q7 podName:f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59 nodeName:}" failed. No retries permitted until 2026-07-09 04:32:17.837208859 +0000 UTC m=+10.088732429 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-886q7" (UniqueName: "kubernetes.io/projected/f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59-kube-api-access-886q7") pod "network-check-target-bgsl7" (UID: "f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:32:14.191786 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:14.190463 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bgsl7" Jul 09 04:32:14.191786 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:14.190592 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bgsl7" podUID="f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59" Jul 09 04:32:14.191786 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:14.190463 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpx47" Jul 09 04:32:14.191786 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:14.190759 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpx47" podUID="dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa" Jul 09 04:32:14.191786 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:14.190813 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zl4tp" Jul 09 04:32:14.191786 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:14.190886 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zl4tp" podUID="d0a12812-bbc1-4b88-a2f2-858d22d67e64" Jul 09 04:32:16.189697 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:16.189604 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpx47" Jul 09 04:32:16.189697 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:16.189634 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bgsl7" Jul 09 04:32:16.189697 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:16.189656 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zl4tp" Jul 09 04:32:16.190255 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:16.189756 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpx47" podUID="dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa" Jul 09 04:32:16.190255 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:16.189826 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bgsl7" podUID="f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59" Jul 09 04:32:16.190255 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:16.189899 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zl4tp" podUID="d0a12812-bbc1-4b88-a2f2-858d22d67e64" Jul 09 04:32:17.766711 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:17.766248 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa-metrics-certs\") pod \"network-metrics-daemon-gpx47\" (UID: \"dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa\") " pod="openshift-multus/network-metrics-daemon-gpx47" Jul 09 04:32:17.766711 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:17.766314 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0a12812-bbc1-4b88-a2f2-858d22d67e64-original-pull-secret\") pod \"global-pull-secret-syncer-zl4tp\" (UID: \"d0a12812-bbc1-4b88-a2f2-858d22d67e64\") " pod="kube-system/global-pull-secret-syncer-zl4tp" Jul 09 04:32:17.766711 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:17.766462 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 04:32:17.766711 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:17.766528 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0a12812-bbc1-4b88-a2f2-858d22d67e64-original-pull-secret podName:d0a12812-bbc1-4b88-a2f2-858d22d67e64 nodeName:}" failed. No retries permitted until 2026-07-09 04:32:25.766509344 +0000 UTC m=+18.018032904 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d0a12812-bbc1-4b88-a2f2-858d22d67e64-original-pull-secret") pod "global-pull-secret-syncer-zl4tp" (UID: "d0a12812-bbc1-4b88-a2f2-858d22d67e64") : object "kube-system"/"original-pull-secret" not registered Jul 09 04:32:17.766711 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:17.766550 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:32:17.766711 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:17.766625 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa-metrics-certs podName:dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa nodeName:}" failed. No retries permitted until 2026-07-09 04:32:25.76660683 +0000 UTC m=+18.018130393 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa-metrics-certs") pod "network-metrics-daemon-gpx47" (UID: "dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:32:17.867714 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:17.867634 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-886q7\" (UniqueName: \"kubernetes.io/projected/f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59-kube-api-access-886q7\") pod \"network-check-target-bgsl7\" (UID: \"f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59\") " pod="openshift-network-diagnostics/network-check-target-bgsl7" Jul 09 04:32:17.867901 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:17.867778 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 04:32:17.867901 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:17.867804 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 04:32:17.867901 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:17.867818 2565 projected.go:196] Error preparing data for projected volume kube-api-access-886q7 for pod openshift-network-diagnostics/network-check-target-bgsl7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:32:17.867901 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:17.867879 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59-kube-api-access-886q7 podName:f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59 nodeName:}" failed. No retries permitted until 2026-07-09 04:32:25.867859232 +0000 UTC m=+18.119382809 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-886q7" (UniqueName: "kubernetes.io/projected/f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59-kube-api-access-886q7") pod "network-check-target-bgsl7" (UID: "f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:32:18.190473 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:18.190394 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zl4tp" Jul 09 04:32:18.190661 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:18.190513 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zl4tp" podUID="d0a12812-bbc1-4b88-a2f2-858d22d67e64" Jul 09 04:32:18.191018 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:18.190880 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bgsl7" Jul 09 04:32:18.191018 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:18.190978 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bgsl7" podUID="f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59" Jul 09 04:32:18.191018 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:18.191001 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpx47" Jul 09 04:32:18.191225 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:18.191095 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpx47" podUID="dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa" Jul 09 04:32:20.196561 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:20.196170 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bgsl7" Jul 09 04:32:20.196561 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:20.196394 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bgsl7" podUID="f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59" Jul 09 04:32:20.196561 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:20.196475 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zl4tp" Jul 09 04:32:20.197096 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:20.196736 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zl4tp" podUID="d0a12812-bbc1-4b88-a2f2-858d22d67e64" Jul 09 04:32:20.197096 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:20.196773 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpx47" Jul 09 04:32:20.197096 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:20.196957 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpx47" podUID="dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa" Jul 09 04:32:22.192935 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:22.192710 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bgsl7" Jul 09 04:32:22.193275 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:22.192751 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zl4tp" Jul 09 04:32:22.193275 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:22.192973 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bgsl7" podUID="f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59" Jul 09 04:32:22.193275 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:22.192780 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpx47" Jul 09 04:32:22.193275 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:22.193034 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zl4tp" podUID="d0a12812-bbc1-4b88-a2f2-858d22d67e64" Jul 09 04:32:22.193275 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:22.193131 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpx47" podUID="dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa" Jul 09 04:32:22.391034 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:22.390983 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" event={"ID":"b6b791a3-d352-45b2-8394-6dae85f91c7f","Type":"ContainerStarted","Data":"d4564a86a5b180b507ccc3912e5074c910115930e0bcae92df13d076e86813c5"} Jul 09 04:32:22.393127 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:22.393094 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-j7zhv" event={"ID":"227fe7b3-97ee-435b-9e07-15cf67dd2beb","Type":"ContainerStarted","Data":"76565d9390693c8140d547604ac10796d92aeb1ded5fa1b8b14d51e3c5d27cf3"} Jul 09 04:32:22.395418 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:22.395389 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xzgbm" event={"ID":"7b2795d7-ba4b-46ea-bd2c-6b9d061e9746","Type":"ContainerStarted","Data":"dcb232f10b9fc88a2d0f40999f27450ceae816b5379e2fa998778aaba5689e9c"} Jul 09 04:32:22.397098 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:22.397068 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fltwp" event={"ID":"97d22a8e-85f8-4d3a-8fbf-cdff7581f85e","Type":"ContainerStarted","Data":"87ace3566d965444e594498c2072bdb25781731b0ac02f207546a78af612b674"} Jul 09 04:32:22.398940 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:22.398909 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cmpgg" event={"ID":"af45d649-376b-4efd-ad4c-9b5b3c28ab4d","Type":"ContainerStarted","Data":"9c50a3826db9717f3d8d53f7d21986eecdc07750744caa65d088589b074c4175"} Jul 09 04:32:22.400852 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:22.400795 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g7j6" event={"ID":"ea2c59ff-64a0-4a8b-b179-6dfbf26be5df","Type":"ContainerStarted","Data":"b8e73e9882a5cd22b8c58c8e22a94d470b6ea1c16c1413fed83fad7c6f51e8bd"} Jul 09 04:32:22.406443 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:22.406396 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-184.ec2.internal" podStartSLOduration=13.406353842 podStartE2EDuration="13.406353842s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:32:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:32:13.385458199 +0000 UTC m=+5.636981780" watchObservedRunningTime="2026-07-09 04:32:22.406353842 +0000 UTC m=+14.657877413" Jul 09 04:32:22.407423 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:22.407368 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-vkz4b" podStartSLOduration=4.005916124 podStartE2EDuration="14.407353599s" totalImagesPullingTime="10.401437475s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:32:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:32:22.406524366 +0000 UTC m=+14.658047936" watchObservedRunningTime="2026-07-09 04:32:22.407353599 +0000 UTC m=+14.658877176" Jul 09 04:32:22.420026 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:22.419981 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-xzgbm" podStartSLOduration=4.004823517 podStartE2EDuration="14.419963276s" totalImagesPullingTime="10.415139759s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:32:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:32:22.419814001 +0000 UTC m=+14.671337583" watchObservedRunningTime="2026-07-09 04:32:22.419963276 +0000 UTC m=+14.671486855" Jul 09 04:32:22.434169 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:22.434110 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-fltwp" podStartSLOduration=4.033287388 podStartE2EDuration="14.434091126s" totalImagesPullingTime="10.400803738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:32:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:32:22.433539934 +0000 UTC m=+14.685063511" watchObservedRunningTime="2026-07-09 04:32:22.434091126 +0000 UTC m=+14.685614704" Jul 09 04:32:22.451670 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:22.451571 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-j7zhv" podStartSLOduration=4.047778677 podStartE2EDuration="14.451550034s" totalImagesPullingTime="10.403771357s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:32:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:32:22.450965193 +0000 UTC m=+14.702488770" watchObservedRunningTime="2026-07-09 04:32:22.451550034 +0000 UTC m=+14.703073613" Jul 09 04:32:22.864943 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:22.864855 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-fltwp" Jul 09 04:32:22.865727 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:22.865700 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-fltwp" Jul 09 04:32:23.404934 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:23.404902 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bzk6k" event={"ID":"22077c07-57a6-4027-acae-197295ac8f18","Type":"ContainerStarted","Data":"7dc828d228775e62548e8070fe25a3c4ab953a00e69eb0dcfb8dc3ec0a7be43a"} Jul 09 04:32:23.405899 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:23.405832 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-fltwp" Jul 09 04:32:23.406401 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:23.406383 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-fltwp" Jul 09 04:32:23.420104 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:23.420053 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bzk6k" podStartSLOduration=5.017979491 podStartE2EDuration="15.420036554s" totalImagesPullingTime="10.402057063s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:32:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:32:23.41977667 +0000 UTC m=+15.671300248" watchObservedRunningTime="2026-07-09 04:32:23.420036554 +0000 UTC m=+15.671560131" Jul 09 04:32:24.189378 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:24.189322 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zl4tp" Jul 09 04:32:24.189553 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:24.189324 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpx47" Jul 09 04:32:24.189553 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:24.189462 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zl4tp" podUID="d0a12812-bbc1-4b88-a2f2-858d22d67e64" Jul 09 04:32:24.189553 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:24.189498 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bgsl7" Jul 09 04:32:24.189718 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:24.189570 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpx47" podUID="dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa" Jul 09 04:32:24.189718 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:24.189654 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bgsl7" podUID="f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59" Jul 09 04:32:25.830200 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:25.830158 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa-metrics-certs\") pod \"network-metrics-daemon-gpx47\" (UID: \"dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa\") " pod="openshift-multus/network-metrics-daemon-gpx47" Jul 09 04:32:25.830907 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:25.830218 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0a12812-bbc1-4b88-a2f2-858d22d67e64-original-pull-secret\") pod \"global-pull-secret-syncer-zl4tp\" (UID: \"d0a12812-bbc1-4b88-a2f2-858d22d67e64\") " pod="kube-system/global-pull-secret-syncer-zl4tp" Jul 09 04:32:25.830907 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:25.830364 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:32:25.830907 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:25.830407 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 04:32:25.830907 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:25.830444 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa-metrics-certs podName:dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa nodeName:}" failed. No retries permitted until 2026-07-09 04:32:41.830422752 +0000 UTC m=+34.081946329 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa-metrics-certs") pod "network-metrics-daemon-gpx47" (UID: "dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:32:25.830907 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:25.830469 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0a12812-bbc1-4b88-a2f2-858d22d67e64-original-pull-secret podName:d0a12812-bbc1-4b88-a2f2-858d22d67e64 nodeName:}" failed. No retries permitted until 2026-07-09 04:32:41.830456328 +0000 UTC m=+34.081979884 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d0a12812-bbc1-4b88-a2f2-858d22d67e64-original-pull-secret") pod "global-pull-secret-syncer-zl4tp" (UID: "d0a12812-bbc1-4b88-a2f2-858d22d67e64") : object "kube-system"/"original-pull-secret" not registered Jul 09 04:32:25.931662 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:25.931628 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-886q7\" (UniqueName: \"kubernetes.io/projected/f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59-kube-api-access-886q7\") pod \"network-check-target-bgsl7\" (UID: \"f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59\") " pod="openshift-network-diagnostics/network-check-target-bgsl7" Jul 09 04:32:25.931856 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:25.931816 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 04:32:25.931856 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:25.931841 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 04:32:25.931856 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:25.931854 2565 projected.go:196] Error preparing data for projected volume kube-api-access-886q7 for pod openshift-network-diagnostics/network-check-target-bgsl7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:32:25.932012 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:25.931922 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59-kube-api-access-886q7 podName:f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59 nodeName:}" failed. No retries permitted until 2026-07-09 04:32:41.931902367 +0000 UTC m=+34.183425940 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-886q7" (UniqueName: "kubernetes.io/projected/f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59-kube-api-access-886q7") pod "network-check-target-bgsl7" (UID: "f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:32:26.194027 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:26.193998 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bgsl7" Jul 09 04:32:26.194027 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:26.194029 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zl4tp" Jul 09 04:32:26.194269 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:26.194103 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpx47" Jul 09 04:32:26.194269 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:26.194110 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bgsl7" podUID="f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59" Jul 09 04:32:26.194269 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:26.194159 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zl4tp" podUID="d0a12812-bbc1-4b88-a2f2-858d22d67e64" Jul 09 04:32:26.194269 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:26.194239 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpx47" podUID="dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa" Jul 09 04:32:26.413817 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:26.413779 2565 generic.go:356] "Generic (PLEG): container finished" podID="af45d649-376b-4efd-ad4c-9b5b3c28ab4d" containerID="9c50a3826db9717f3d8d53f7d21986eecdc07750744caa65d088589b074c4175" exitCode=0 Jul 09 04:32:26.414005 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:26.413849 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cmpgg" event={"ID":"af45d649-376b-4efd-ad4c-9b5b3c28ab4d","Type":"ContainerDied","Data":"9c50a3826db9717f3d8d53f7d21986eecdc07750744caa65d088589b074c4175"} Jul 09 04:32:28.190611 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:28.190579 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpx47" Jul 09 04:32:28.191204 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:28.190635 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bgsl7" Jul 09 04:32:28.191204 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:28.190709 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bgsl7" podUID="f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59" Jul 09 04:32:28.191204 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:28.190723 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zl4tp" Jul 09 04:32:28.191204 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:28.190832 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zl4tp" podUID="d0a12812-bbc1-4b88-a2f2-858d22d67e64" Jul 09 04:32:28.191204 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:28.190902 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpx47" podUID="dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa" Jul 09 04:32:30.193519 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:30.193489 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zl4tp" Jul 09 04:32:30.193946 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:30.193604 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bgsl7" Jul 09 04:32:30.193946 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:30.193609 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zl4tp" podUID="d0a12812-bbc1-4b88-a2f2-858d22d67e64" Jul 09 04:32:30.193946 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:30.193641 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpx47" Jul 09 04:32:30.193946 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:30.193734 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bgsl7" podUID="f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59" Jul 09 04:32:30.193946 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:30.193829 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpx47" podUID="dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa" Jul 09 04:32:30.503511 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:30.503282 2565 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 09 04:32:31.158870 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:31.158730 2565 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T04:32:30.503484198Z","UUID":"e1eb141a-d956-4968-946b-e4adccd20a00","Handler":null,"Name":"","Endpoint":""} Jul 09 04:32:31.160694 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:31.160469 2565 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 09 04:32:31.160822 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:31.160712 2565 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 09 04:32:31.428551 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:31.428474 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" event={"ID":"52c0430b-efce-47bf-9a4f-2428de23d349","Type":"ContainerStarted","Data":"6db0ab4d70b1d0a1df3664147fa250d4b4dc14f1b6999def999128d646bb5e7a"} Jul 09 04:32:31.428981 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:31.428586 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" event={"ID":"52c0430b-efce-47bf-9a4f-2428de23d349","Type":"ContainerStarted","Data":"44f255a07cbe7e0f3fea04a081a417e016d0ef256f0f102c265b741f48763c64"} Jul 09 04:32:31.428981 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:31.428633 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" event={"ID":"52c0430b-efce-47bf-9a4f-2428de23d349","Type":"ContainerStarted","Data":"c60ba3515de8fb71280336f42141bf381c50dee897f45615eebc1bdc359ba63d"} Jul 09 04:32:31.428981 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:31.428663 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" event={"ID":"52c0430b-efce-47bf-9a4f-2428de23d349","Type":"ContainerStarted","Data":"9cd2d3d5387ef3b698195c71c9a30877d6890561e8356490320ea6e7f22542d2"} Jul 09 04:32:31.428981 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:31.428694 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" event={"ID":"52c0430b-efce-47bf-9a4f-2428de23d349","Type":"ContainerStarted","Data":"8056d56a1c17fcc0f00c6f17f20e1d15f6660ce032554b04d47c084f79b01c8a"} Jul 09 04:32:31.428981 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:31.428737 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" event={"ID":"52c0430b-efce-47bf-9a4f-2428de23d349","Type":"ContainerStarted","Data":"fc378d199d5011182065fc2eb1261eede0191d5ecea82cbf878cda0207661a77"} Jul 09 04:32:31.430209 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:31.430186 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g7j6" event={"ID":"ea2c59ff-64a0-4a8b-b179-6dfbf26be5df","Type":"ContainerStarted","Data":"683b33ee81f1bfd6c4f8e805796830e9d4fa329f9f4e60136ba88a9613c306b3"} Jul 09 04:32:31.431584 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:31.431561 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2tgfm" event={"ID":"2b4a7d33-f06c-489c-ab25-b7b5b109d33c","Type":"ContainerStarted","Data":"99cf2b7f57d3e72475a7dbc64296248656f16fdc200019aeffdcb78c60701eda"} Jul 09 04:32:31.448178 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:31.448129 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-2tgfm" podStartSLOduration=4.423946545 podStartE2EDuration="23.448115573s" totalImagesPullingTime="19.024169028s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:32:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:32:31.447729464 +0000 UTC m=+23.699253042" watchObservedRunningTime="2026-07-09 04:32:31.448115573 +0000 UTC m=+23.699639151" Jul 09 04:32:32.189441 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:32.189410 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bgsl7" Jul 09 04:32:32.189723 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:32.189410 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpx47" Jul 09 04:32:32.189723 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:32.189529 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zl4tp" Jul 09 04:32:32.189723 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:32.189535 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bgsl7" podUID="f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59" Jul 09 04:32:32.189723 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:32.189615 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpx47" podUID="dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa" Jul 09 04:32:32.189723 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:32.189685 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zl4tp" podUID="d0a12812-bbc1-4b88-a2f2-858d22d67e64" Jul 09 04:32:33.439130 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:33.439095 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" event={"ID":"52c0430b-efce-47bf-9a4f-2428de23d349","Type":"ContainerStarted","Data":"f1826a0915487d4ef4264dbf65235f250c58acb724df1fe2c2bc09b120914393"} Jul 09 04:32:33.441192 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:33.441167 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g7j6" event={"ID":"ea2c59ff-64a0-4a8b-b179-6dfbf26be5df","Type":"ContainerStarted","Data":"57c7b67397216a4afa2b686ea7e1b9c2ee7462d1714875aff8b3464a00f7b706"} Jul 09 04:32:33.458080 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:33.458034 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4g7j6" podStartSLOduration=4.188845401 podStartE2EDuration="25.458016443s" totalImagesPullingTime="21.269171042s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:32:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:32:33.45711368 +0000 UTC m=+25.708637257" watchObservedRunningTime="2026-07-09 04:32:33.458016443 +0000 UTC m=+25.709540022" Jul 09 04:32:34.189294 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:34.189261 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpx47" Jul 09 04:32:34.189504 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:34.189262 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bgsl7" Jul 09 04:32:34.189504 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:34.189406 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpx47" podUID="dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa" Jul 09 04:32:34.189504 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:34.189264 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zl4tp" Jul 09 04:32:34.189504 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:34.189465 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bgsl7" podUID="f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59" Jul 09 04:32:34.189722 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:34.189530 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zl4tp" podUID="d0a12812-bbc1-4b88-a2f2-858d22d67e64" Jul 09 04:32:35.377935 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:35.377892 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:35.449966 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:35.449934 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" event={"ID":"52c0430b-efce-47bf-9a4f-2428de23d349","Type":"ContainerStarted","Data":"f925f2b7dbbaf89ac191910de476a210542d9d70c83deeaaae95155065bd844d"} Jul 09 04:32:35.450230 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:35.450213 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:35.451950 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:35.451927 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cmpgg" event={"ID":"af45d649-376b-4efd-ad4c-9b5b3c28ab4d","Type":"ContainerStarted","Data":"55d98715c72cde2f362d17df163351d842109b05f2b07acd5f5766da31a69bf2"} Jul 09 04:32:35.462856 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:35.462820 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:35.465562 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:35.465532 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:35.476907 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:35.476862 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" podStartSLOduration=8.450643762 podStartE2EDuration="27.476849711s" totalImagesPullingTime="19.026205949s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:32:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:32:35.475616718 +0000 UTC m=+27.727140322" watchObservedRunningTime="2026-07-09 04:32:35.476849711 +0000 UTC m=+27.728373288" Jul 09 04:32:35.534377 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:35.534179 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:35.872537 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:35.872499 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:36.189680 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:36.189650 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zl4tp" Jul 09 04:32:36.189680 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:36.189688 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bgsl7" Jul 09 04:32:36.189949 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:36.189655 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpx47" Jul 09 04:32:36.189949 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:36.189754 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zl4tp" podUID="d0a12812-bbc1-4b88-a2f2-858d22d67e64" Jul 09 04:32:36.189949 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:36.189875 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpx47" podUID="dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa" Jul 09 04:32:36.190050 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:36.189978 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bgsl7" podUID="f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59" Jul 09 04:32:36.456919 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:36.456836 2565 generic.go:356] "Generic (PLEG): container finished" podID="af45d649-376b-4efd-ad4c-9b5b3c28ab4d" containerID="55d98715c72cde2f362d17df163351d842109b05f2b07acd5f5766da31a69bf2" exitCode=0 Jul 09 04:32:36.457392 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:36.457348 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cmpgg" event={"ID":"af45d649-376b-4efd-ad4c-9b5b3c28ab4d","Type":"ContainerDied","Data":"55d98715c72cde2f362d17df163351d842109b05f2b07acd5f5766da31a69bf2"} Jul 09 04:32:36.457496 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:36.457420 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:36.457496 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:36.457489 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:36.474094 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:36.474072 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:32:37.117270 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:37.117240 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zl4tp"] Jul 09 04:32:37.117447 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:37.117333 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zl4tp" Jul 09 04:32:37.117447 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:37.117432 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zl4tp" podUID="d0a12812-bbc1-4b88-a2f2-858d22d67e64" Jul 09 04:32:37.129427 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:37.129397 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bgsl7"] Jul 09 04:32:37.129570 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:37.129486 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bgsl7" Jul 09 04:32:37.129570 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:37.129559 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bgsl7" podUID="f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59" Jul 09 04:32:37.141614 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:37.141587 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gpx47"] Jul 09 04:32:37.141730 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:37.141677 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpx47" Jul 09 04:32:37.141778 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:37.141755 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpx47" podUID="dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa" Jul 09 04:32:38.473663 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:38.473627 2565 generic.go:356] "Generic (PLEG): container finished" podID="af45d649-376b-4efd-ad4c-9b5b3c28ab4d" containerID="311f391c89aabdf29f3ba36a5b1031f6f8016dfb83cc1cb81cfb3c73e368cbc9" exitCode=0 Jul 09 04:32:38.474029 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:38.473695 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cmpgg" event={"ID":"af45d649-376b-4efd-ad4c-9b5b3c28ab4d","Type":"ContainerDied","Data":"311f391c89aabdf29f3ba36a5b1031f6f8016dfb83cc1cb81cfb3c73e368cbc9"} Jul 09 04:32:39.189397 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:39.189373 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpx47" Jul 09 04:32:39.189397 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:39.189384 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bgsl7" Jul 09 04:32:39.189571 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:39.189375 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zl4tp" Jul 09 04:32:39.189571 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:39.189509 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpx47" podUID="dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa" Jul 09 04:32:39.189680 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:39.189630 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zl4tp" podUID="d0a12812-bbc1-4b88-a2f2-858d22d67e64" Jul 09 04:32:39.189734 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:39.189710 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bgsl7" podUID="f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59" Jul 09 04:32:40.481216 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:40.481029 2565 generic.go:356] "Generic (PLEG): container finished" podID="af45d649-376b-4efd-ad4c-9b5b3c28ab4d" containerID="220b0603fe5fe11a5d9927245d81fa12981269f4686db044667157a22c08deae" exitCode=0 Jul 09 04:32:40.481622 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:40.481100 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cmpgg" event={"ID":"af45d649-376b-4efd-ad4c-9b5b3c28ab4d","Type":"ContainerDied","Data":"220b0603fe5fe11a5d9927245d81fa12981269f4686db044667157a22c08deae"} Jul 09 04:32:41.190274 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:41.190234 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bgsl7" Jul 09 04:32:41.190274 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:41.190265 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zl4tp" Jul 09 04:32:41.190523 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:41.190266 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpx47" Jul 09 04:32:41.190523 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:41.190379 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bgsl7" podUID="f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59" Jul 09 04:32:41.190523 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:41.190448 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zl4tp" podUID="d0a12812-bbc1-4b88-a2f2-858d22d67e64" Jul 09 04:32:41.190629 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:41.190584 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gpx47" podUID="dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa" Jul 09 04:32:41.853244 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:41.853210 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa-metrics-certs\") pod \"network-metrics-daemon-gpx47\" (UID: \"dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa\") " pod="openshift-multus/network-metrics-daemon-gpx47" Jul 09 04:32:41.853842 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:41.853264 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0a12812-bbc1-4b88-a2f2-858d22d67e64-original-pull-secret\") pod \"global-pull-secret-syncer-zl4tp\" (UID: \"d0a12812-bbc1-4b88-a2f2-858d22d67e64\") " pod="kube-system/global-pull-secret-syncer-zl4tp" Jul 09 04:32:41.853842 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:41.853401 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 04:32:41.853842 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:41.853409 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:32:41.853842 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:41.853468 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa-metrics-certs podName:dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa nodeName:}" failed. No retries permitted until 2026-07-09 04:33:13.853451389 +0000 UTC m=+66.104974944 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa-metrics-certs") pod "network-metrics-daemon-gpx47" (UID: "dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:32:41.853842 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:41.853491 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0a12812-bbc1-4b88-a2f2-858d22d67e64-original-pull-secret podName:d0a12812-bbc1-4b88-a2f2-858d22d67e64 nodeName:}" failed. No retries permitted until 2026-07-09 04:33:13.853478785 +0000 UTC m=+66.105002342 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d0a12812-bbc1-4b88-a2f2-858d22d67e64-original-pull-secret") pod "global-pull-secret-syncer-zl4tp" (UID: "d0a12812-bbc1-4b88-a2f2-858d22d67e64") : object "kube-system"/"original-pull-secret" not registered Jul 09 04:32:41.954237 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:41.954198 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-886q7\" (UniqueName: \"kubernetes.io/projected/f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59-kube-api-access-886q7\") pod \"network-check-target-bgsl7\" (UID: \"f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59\") " pod="openshift-network-diagnostics/network-check-target-bgsl7" Jul 09 04:32:41.954437 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:41.954381 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 04:32:41.954437 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:41.954409 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 04:32:41.954437 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:41.954422 2565 projected.go:196] Error preparing data for projected volume kube-api-access-886q7 for pod openshift-network-diagnostics/network-check-target-bgsl7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:32:41.954596 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:41.954490 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59-kube-api-access-886q7 podName:f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59 nodeName:}" failed. No retries permitted until 2026-07-09 04:33:13.954470068 +0000 UTC m=+66.205993656 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-886q7" (UniqueName: "kubernetes.io/projected/f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59-kube-api-access-886q7") pod "network-check-target-bgsl7" (UID: "f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:32:42.985354 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:42.985306 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:43.038237 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.038148 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:43.051841 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.051813 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-184.ec2.internal" event="NodeReady" Jul 09 04:32:43.052020 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.051963 2565 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 09 04:32:43.085024 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.084991 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-75979d6b76-prn9d"] Jul 09 04:32:43.088669 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.088649 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75979d6b76-prn9d" Jul 09 04:32:43.091290 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.091099 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 09 04:32:43.091290 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.091175 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 09 04:32:43.091290 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.091278 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 09 04:32:43.091538 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.091284 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8hcw8\"" Jul 09 04:32:43.097365 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.097253 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-wbngq"] Jul 09 04:32:43.101641 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.101621 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wbngq" Jul 09 04:32:43.101641 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.101639 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 09 04:32:43.101795 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.101735 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-ct9t6"] Jul 09 04:32:43.104513 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.104493 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 09 04:32:43.104677 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.104540 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 09 04:32:43.104777 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.104586 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9rkp9\"" Jul 09 04:32:43.104777 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.104605 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 09 04:32:43.105837 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.105818 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75979d6b76-prn9d"] Jul 09 04:32:43.105953 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.105937 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ct9t6" Jul 09 04:32:43.108211 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.108171 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 09 04:32:43.108322 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.108296 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-58ffk\"" Jul 09 04:32:43.108478 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.108462 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 09 04:32:43.111645 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.111626 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wbngq"] Jul 09 04:32:43.118990 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.118958 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ct9t6"] Jul 09 04:32:43.163410 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.163376 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1db69bb4-f741-480b-9608-03c97923a6c1-image-registry-private-configuration\") pod \"image-registry-75979d6b76-prn9d\" (UID: \"1db69bb4-f741-480b-9608-03c97923a6c1\") " pod="openshift-image-registry/image-registry-75979d6b76-prn9d" Jul 09 04:32:43.163410 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.163412 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1db69bb4-f741-480b-9608-03c97923a6c1-installation-pull-secrets\") pod \"image-registry-75979d6b76-prn9d\" (UID: \"1db69bb4-f741-480b-9608-03c97923a6c1\") " pod="openshift-image-registry/image-registry-75979d6b76-prn9d" Jul 09 04:32:43.163621 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.163478 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/964ab4da-c25d-46bd-97df-ffb2cd5b820b-cert\") pod \"ingress-canary-wbngq\" (UID: \"964ab4da-c25d-46bd-97df-ffb2cd5b820b\") " pod="openshift-ingress-canary/ingress-canary-wbngq" Jul 09 04:32:43.163621 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.163543 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wvhp\" (UniqueName: \"kubernetes.io/projected/1db69bb4-f741-480b-9608-03c97923a6c1-kube-api-access-9wvhp\") pod \"image-registry-75979d6b76-prn9d\" (UID: \"1db69bb4-f741-480b-9608-03c97923a6c1\") " pod="openshift-image-registry/image-registry-75979d6b76-prn9d" Jul 09 04:32:43.163621 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.163579 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1db69bb4-f741-480b-9608-03c97923a6c1-registry-tls\") pod \"image-registry-75979d6b76-prn9d\" (UID: \"1db69bb4-f741-480b-9608-03c97923a6c1\") " pod="openshift-image-registry/image-registry-75979d6b76-prn9d" Jul 09 04:32:43.163621 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.163599 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1db69bb4-f741-480b-9608-03c97923a6c1-ca-trust-extracted\") pod \"image-registry-75979d6b76-prn9d\" (UID: \"1db69bb4-f741-480b-9608-03c97923a6c1\") " pod="openshift-image-registry/image-registry-75979d6b76-prn9d" Jul 09 04:32:43.163621 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.163619 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1db69bb4-f741-480b-9608-03c97923a6c1-trusted-ca\") pod \"image-registry-75979d6b76-prn9d\" (UID: \"1db69bb4-f741-480b-9608-03c97923a6c1\") " pod="openshift-image-registry/image-registry-75979d6b76-prn9d" Jul 09 04:32:43.163820 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.163647 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1db69bb4-f741-480b-9608-03c97923a6c1-bound-sa-token\") pod \"image-registry-75979d6b76-prn9d\" (UID: \"1db69bb4-f741-480b-9608-03c97923a6c1\") " pod="openshift-image-registry/image-registry-75979d6b76-prn9d" Jul 09 04:32:43.163820 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.163732 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1db69bb4-f741-480b-9608-03c97923a6c1-registry-certificates\") pod \"image-registry-75979d6b76-prn9d\" (UID: \"1db69bb4-f741-480b-9608-03c97923a6c1\") " pod="openshift-image-registry/image-registry-75979d6b76-prn9d" Jul 09 04:32:43.163820 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.163789 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdl5c\" (UniqueName: \"kubernetes.io/projected/964ab4da-c25d-46bd-97df-ffb2cd5b820b-kube-api-access-wdl5c\") pod \"ingress-canary-wbngq\" (UID: \"964ab4da-c25d-46bd-97df-ffb2cd5b820b\") " pod="openshift-ingress-canary/ingress-canary-wbngq" Jul 09 04:32:43.189673 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.189642 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zl4tp" Jul 09 04:32:43.189673 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.189679 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpx47" Jul 09 04:32:43.189905 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.189744 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bgsl7" Jul 09 04:32:43.192868 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.192698 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 04:32:43.192868 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.192738 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 04:32:43.192868 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.192753 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 04:32:43.192868 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.192753 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 04:32:43.192868 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.192810 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bx88j\"" Jul 09 04:32:43.192868 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.192698 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8mcbs\"" Jul 09 04:32:43.264522 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.264489 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1db69bb4-f741-480b-9608-03c97923a6c1-trusted-ca\") pod \"image-registry-75979d6b76-prn9d\" (UID: \"1db69bb4-f741-480b-9608-03c97923a6c1\") " pod="openshift-image-registry/image-registry-75979d6b76-prn9d" Jul 09 04:32:43.264522 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.264526 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766-config-volume\") pod \"dns-default-ct9t6\" (UID: \"7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766\") " pod="openshift-dns/dns-default-ct9t6" Jul 09 04:32:43.264749 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.264557 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1db69bb4-f741-480b-9608-03c97923a6c1-bound-sa-token\") pod \"image-registry-75979d6b76-prn9d\" (UID: \"1db69bb4-f741-480b-9608-03c97923a6c1\") " pod="openshift-image-registry/image-registry-75979d6b76-prn9d" Jul 09 04:32:43.264749 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.264684 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1db69bb4-f741-480b-9608-03c97923a6c1-registry-certificates\") pod \"image-registry-75979d6b76-prn9d\" (UID: \"1db69bb4-f741-480b-9608-03c97923a6c1\") " pod="openshift-image-registry/image-registry-75979d6b76-prn9d" Jul 09 04:32:43.264749 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.264737 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdl5c\" (UniqueName: \"kubernetes.io/projected/964ab4da-c25d-46bd-97df-ffb2cd5b820b-kube-api-access-wdl5c\") pod \"ingress-canary-wbngq\" (UID: \"964ab4da-c25d-46bd-97df-ffb2cd5b820b\") " pod="openshift-ingress-canary/ingress-canary-wbngq" Jul 09 04:32:43.264945 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.264782 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1db69bb4-f741-480b-9608-03c97923a6c1-image-registry-private-configuration\") pod \"image-registry-75979d6b76-prn9d\" (UID: \"1db69bb4-f741-480b-9608-03c97923a6c1\") " pod="openshift-image-registry/image-registry-75979d6b76-prn9d" Jul 09 04:32:43.264945 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.264810 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766-tmp-dir\") pod \"dns-default-ct9t6\" (UID: \"7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766\") " pod="openshift-dns/dns-default-ct9t6" Jul 09 04:32:43.264945 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.264844 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1db69bb4-f741-480b-9608-03c97923a6c1-installation-pull-secrets\") pod \"image-registry-75979d6b76-prn9d\" (UID: \"1db69bb4-f741-480b-9608-03c97923a6c1\") " pod="openshift-image-registry/image-registry-75979d6b76-prn9d" Jul 09 04:32:43.264945 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.264880 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/964ab4da-c25d-46bd-97df-ffb2cd5b820b-cert\") pod \"ingress-canary-wbngq\" (UID: \"964ab4da-c25d-46bd-97df-ffb2cd5b820b\") " pod="openshift-ingress-canary/ingress-canary-wbngq" Jul 09 04:32:43.264945 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.264921 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wvhp\" (UniqueName: \"kubernetes.io/projected/1db69bb4-f741-480b-9608-03c97923a6c1-kube-api-access-9wvhp\") pod \"image-registry-75979d6b76-prn9d\" (UID: \"1db69bb4-f741-480b-9608-03c97923a6c1\") " pod="openshift-image-registry/image-registry-75979d6b76-prn9d" Jul 09 04:32:43.264945 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.264946 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766-metrics-tls\") pod \"dns-default-ct9t6\" (UID: \"7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766\") " pod="openshift-dns/dns-default-ct9t6" Jul 09 04:32:43.265216 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.264979 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8sjq\" (UniqueName: \"kubernetes.io/projected/7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766-kube-api-access-b8sjq\") pod \"dns-default-ct9t6\" (UID: \"7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766\") " pod="openshift-dns/dns-default-ct9t6" Jul 09 04:32:43.265216 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.265007 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1db69bb4-f741-480b-9608-03c97923a6c1-registry-tls\") pod \"image-registry-75979d6b76-prn9d\" (UID: \"1db69bb4-f741-480b-9608-03c97923a6c1\") " pod="openshift-image-registry/image-registry-75979d6b76-prn9d" Jul 09 04:32:43.265216 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.265034 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1db69bb4-f741-480b-9608-03c97923a6c1-ca-trust-extracted\") pod \"image-registry-75979d6b76-prn9d\" (UID: \"1db69bb4-f741-480b-9608-03c97923a6c1\") " pod="openshift-image-registry/image-registry-75979d6b76-prn9d" Jul 09 04:32:43.265381 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:43.265237 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 04:32:43.265381 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:43.265257 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75979d6b76-prn9d: secret "image-registry-tls" not found Jul 09 04:32:43.265381 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.265285 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1db69bb4-f741-480b-9608-03c97923a6c1-registry-certificates\") pod \"image-registry-75979d6b76-prn9d\" (UID: \"1db69bb4-f741-480b-9608-03c97923a6c1\") " pod="openshift-image-registry/image-registry-75979d6b76-prn9d" Jul 09 04:32:43.265381 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:43.265323 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1db69bb4-f741-480b-9608-03c97923a6c1-registry-tls podName:1db69bb4-f741-480b-9608-03c97923a6c1 nodeName:}" failed. No retries permitted until 2026-07-09 04:32:43.765301309 +0000 UTC m=+36.016824879 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1db69bb4-f741-480b-9608-03c97923a6c1-registry-tls") pod "image-registry-75979d6b76-prn9d" (UID: "1db69bb4-f741-480b-9608-03c97923a6c1") : secret "image-registry-tls" not found Jul 09 04:32:43.265587 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.265564 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1db69bb4-f741-480b-9608-03c97923a6c1-trusted-ca\") pod \"image-registry-75979d6b76-prn9d\" (UID: \"1db69bb4-f741-480b-9608-03c97923a6c1\") " pod="openshift-image-registry/image-registry-75979d6b76-prn9d" Jul 09 04:32:43.265848 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.265825 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1db69bb4-f741-480b-9608-03c97923a6c1-ca-trust-extracted\") pod \"image-registry-75979d6b76-prn9d\" (UID: \"1db69bb4-f741-480b-9608-03c97923a6c1\") " pod="openshift-image-registry/image-registry-75979d6b76-prn9d" Jul 09 04:32:43.270139 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.270112 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1db69bb4-f741-480b-9608-03c97923a6c1-image-registry-private-configuration\") pod \"image-registry-75979d6b76-prn9d\" (UID: \"1db69bb4-f741-480b-9608-03c97923a6c1\") " pod="openshift-image-registry/image-registry-75979d6b76-prn9d" Jul 09 04:32:43.270139 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.270124 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1db69bb4-f741-480b-9608-03c97923a6c1-installation-pull-secrets\") pod \"image-registry-75979d6b76-prn9d\" (UID: \"1db69bb4-f741-480b-9608-03c97923a6c1\") " pod="openshift-image-registry/image-registry-75979d6b76-prn9d" Jul 09 04:32:43.270299 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.270131 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/964ab4da-c25d-46bd-97df-ffb2cd5b820b-cert\") pod \"ingress-canary-wbngq\" (UID: \"964ab4da-c25d-46bd-97df-ffb2cd5b820b\") " pod="openshift-ingress-canary/ingress-canary-wbngq" Jul 09 04:32:43.273087 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.273064 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdl5c\" (UniqueName: \"kubernetes.io/projected/964ab4da-c25d-46bd-97df-ffb2cd5b820b-kube-api-access-wdl5c\") pod \"ingress-canary-wbngq\" (UID: \"964ab4da-c25d-46bd-97df-ffb2cd5b820b\") " pod="openshift-ingress-canary/ingress-canary-wbngq" Jul 09 04:32:43.273238 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.273217 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1db69bb4-f741-480b-9608-03c97923a6c1-bound-sa-token\") pod \"image-registry-75979d6b76-prn9d\" (UID: \"1db69bb4-f741-480b-9608-03c97923a6c1\") " pod="openshift-image-registry/image-registry-75979d6b76-prn9d" Jul 09 04:32:43.273993 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.273970 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wvhp\" (UniqueName: \"kubernetes.io/projected/1db69bb4-f741-480b-9608-03c97923a6c1-kube-api-access-9wvhp\") pod \"image-registry-75979d6b76-prn9d\" (UID: \"1db69bb4-f741-480b-9608-03c97923a6c1\") " pod="openshift-image-registry/image-registry-75979d6b76-prn9d" Jul 09 04:32:43.365479 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.365382 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766-metrics-tls\") pod \"dns-default-ct9t6\" (UID: \"7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766\") " pod="openshift-dns/dns-default-ct9t6" Jul 09 04:32:43.365479 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.365429 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8sjq\" (UniqueName: \"kubernetes.io/projected/7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766-kube-api-access-b8sjq\") pod \"dns-default-ct9t6\" (UID: \"7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766\") " pod="openshift-dns/dns-default-ct9t6" Jul 09 04:32:43.365479 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.365474 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766-config-volume\") pod \"dns-default-ct9t6\" (UID: \"7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766\") " pod="openshift-dns/dns-default-ct9t6" Jul 09 04:32:43.365757 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:43.365548 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 04:32:43.365757 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:43.365617 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766-metrics-tls podName:7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766 nodeName:}" failed. No retries permitted until 2026-07-09 04:32:43.865598142 +0000 UTC m=+36.117121697 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766-metrics-tls") pod "dns-default-ct9t6" (UID: "7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766") : secret "dns-default-metrics-tls" not found Jul 09 04:32:43.365757 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.365737 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766-tmp-dir\") pod \"dns-default-ct9t6\" (UID: \"7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766\") " pod="openshift-dns/dns-default-ct9t6" Jul 09 04:32:43.366098 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.366077 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766-tmp-dir\") pod \"dns-default-ct9t6\" (UID: \"7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766\") " pod="openshift-dns/dns-default-ct9t6" Jul 09 04:32:43.366172 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.366123 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766-config-volume\") pod \"dns-default-ct9t6\" (UID: \"7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766\") " pod="openshift-dns/dns-default-ct9t6" Jul 09 04:32:43.373908 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.373885 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8sjq\" (UniqueName: \"kubernetes.io/projected/7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766-kube-api-access-b8sjq\") pod \"dns-default-ct9t6\" (UID: \"7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766\") " pod="openshift-dns/dns-default-ct9t6" Jul 09 04:32:43.412962 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.412925 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wbngq" Jul 09 04:32:43.770034 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.769981 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1db69bb4-f741-480b-9608-03c97923a6c1-registry-tls\") pod \"image-registry-75979d6b76-prn9d\" (UID: \"1db69bb4-f741-480b-9608-03c97923a6c1\") " pod="openshift-image-registry/image-registry-75979d6b76-prn9d" Jul 09 04:32:43.770212 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:43.770143 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 04:32:43.770212 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:43.770162 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75979d6b76-prn9d: secret "image-registry-tls" not found Jul 09 04:32:43.770292 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:43.770239 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1db69bb4-f741-480b-9608-03c97923a6c1-registry-tls podName:1db69bb4-f741-480b-9608-03c97923a6c1 nodeName:}" failed. No retries permitted until 2026-07-09 04:32:44.770218144 +0000 UTC m=+37.021741734 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1db69bb4-f741-480b-9608-03c97923a6c1-registry-tls") pod "image-registry-75979d6b76-prn9d" (UID: "1db69bb4-f741-480b-9608-03c97923a6c1") : secret "image-registry-tls" not found Jul 09 04:32:43.871066 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:43.871027 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766-metrics-tls\") pod \"dns-default-ct9t6\" (UID: \"7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766\") " pod="openshift-dns/dns-default-ct9t6" Jul 09 04:32:43.871263 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:43.871227 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 04:32:43.871329 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:43.871310 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766-metrics-tls podName:7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766 nodeName:}" failed. No retries permitted until 2026-07-09 04:32:44.871288531 +0000 UTC m=+37.122812110 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766-metrics-tls") pod "dns-default-ct9t6" (UID: "7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766") : secret "dns-default-metrics-tls" not found Jul 09 04:32:44.515369 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:44.515208 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wbngq"] Jul 09 04:32:44.519177 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:32:44.518977 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod964ab4da_c25d_46bd_97df_ffb2cd5b820b.slice/crio-f924e7b500a872908ae704d0033c6cd6dbb178635ddeeb387eaf8475ec4b0091 WatchSource:0}: Error finding container f924e7b500a872908ae704d0033c6cd6dbb178635ddeeb387eaf8475ec4b0091: Status 404 returned error can't find the container with id f924e7b500a872908ae704d0033c6cd6dbb178635ddeeb387eaf8475ec4b0091 Jul 09 04:32:44.777999 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:44.777963 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1db69bb4-f741-480b-9608-03c97923a6c1-registry-tls\") pod \"image-registry-75979d6b76-prn9d\" (UID: \"1db69bb4-f741-480b-9608-03c97923a6c1\") " pod="openshift-image-registry/image-registry-75979d6b76-prn9d" Jul 09 04:32:44.778166 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:44.778114 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 04:32:44.778166 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:44.778134 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75979d6b76-prn9d: secret "image-registry-tls" not found Jul 09 04:32:44.778231 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:44.778193 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1db69bb4-f741-480b-9608-03c97923a6c1-registry-tls podName:1db69bb4-f741-480b-9608-03c97923a6c1 nodeName:}" failed. No retries permitted until 2026-07-09 04:32:46.778174521 +0000 UTC m=+39.029698096 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1db69bb4-f741-480b-9608-03c97923a6c1-registry-tls") pod "image-registry-75979d6b76-prn9d" (UID: "1db69bb4-f741-480b-9608-03c97923a6c1") : secret "image-registry-tls" not found Jul 09 04:32:44.879100 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:44.879058 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766-metrics-tls\") pod \"dns-default-ct9t6\" (UID: \"7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766\") " pod="openshift-dns/dns-default-ct9t6" Jul 09 04:32:44.879246 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:44.879222 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 04:32:44.879303 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:44.879289 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766-metrics-tls podName:7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766 nodeName:}" failed. No retries permitted until 2026-07-09 04:32:46.879270327 +0000 UTC m=+39.130793882 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766-metrics-tls") pod "dns-default-ct9t6" (UID: "7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766") : secret "dns-default-metrics-tls" not found Jul 09 04:32:45.495083 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:45.495044 2565 generic.go:356] "Generic (PLEG): container finished" podID="af45d649-376b-4efd-ad4c-9b5b3c28ab4d" containerID="f6b19ce817012a50553070fddf811d072391667ac3e539790d63dfd6583e0fc7" exitCode=0 Jul 09 04:32:45.495255 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:45.495128 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cmpgg" event={"ID":"af45d649-376b-4efd-ad4c-9b5b3c28ab4d","Type":"ContainerDied","Data":"f6b19ce817012a50553070fddf811d072391667ac3e539790d63dfd6583e0fc7"} Jul 09 04:32:45.496556 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:45.496451 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wbngq" event={"ID":"964ab4da-c25d-46bd-97df-ffb2cd5b820b","Type":"ContainerStarted","Data":"f924e7b500a872908ae704d0033c6cd6dbb178635ddeeb387eaf8475ec4b0091"} Jul 09 04:32:45.788866 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:45.788788 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:45.889556 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:45.889527 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:46.317316 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:46.317283 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:46.504745 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:46.504713 2565 generic.go:356] "Generic (PLEG): container finished" podID="af45d649-376b-4efd-ad4c-9b5b3c28ab4d" containerID="75e675fd139db0ef9ffff7eb60ca6ae91109267cf2f2d90e48452e4225acf34c" exitCode=0 Jul 09 04:32:46.504745 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:46.504756 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cmpgg" event={"ID":"af45d649-376b-4efd-ad4c-9b5b3c28ab4d","Type":"ContainerDied","Data":"75e675fd139db0ef9ffff7eb60ca6ae91109267cf2f2d90e48452e4225acf34c"} Jul 09 04:32:46.591678 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:46.591592 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-xgznh"] Jul 09 04:32:46.622805 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:46.622715 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xgznh"] Jul 09 04:32:46.622991 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:46.622812 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xgznh" Jul 09 04:32:46.631381 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:46.631354 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 09 04:32:46.631474 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:46.631367 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 09 04:32:46.631682 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:46.631661 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-c4pmv\"" Jul 09 04:32:46.631682 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:46.631677 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 09 04:32:46.635976 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:46.635953 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 09 04:32:46.654396 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:46.653955 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:46.695060 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:46.695029 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/359f3d6e-a08a-487d-a480-a642dad64ec4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xgznh\" (UID: \"359f3d6e-a08a-487d-a480-a642dad64ec4\") " pod="openshift-insights/insights-runtime-extractor-xgznh" Jul 09 04:32:46.695197 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:46.695085 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzqp7\" (UniqueName: \"kubernetes.io/projected/359f3d6e-a08a-487d-a480-a642dad64ec4-kube-api-access-zzqp7\") pod \"insights-runtime-extractor-xgznh\" (UID: \"359f3d6e-a08a-487d-a480-a642dad64ec4\") " pod="openshift-insights/insights-runtime-extractor-xgznh" Jul 09 04:32:46.695197 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:46.695175 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/359f3d6e-a08a-487d-a480-a642dad64ec4-data-volume\") pod \"insights-runtime-extractor-xgznh\" (UID: \"359f3d6e-a08a-487d-a480-a642dad64ec4\") " pod="openshift-insights/insights-runtime-extractor-xgznh" Jul 09 04:32:46.695275 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:46.695217 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/359f3d6e-a08a-487d-a480-a642dad64ec4-crio-socket\") pod \"insights-runtime-extractor-xgznh\" (UID: \"359f3d6e-a08a-487d-a480-a642dad64ec4\") " pod="openshift-insights/insights-runtime-extractor-xgznh" Jul 09 04:32:46.695275 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:46.695249 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/359f3d6e-a08a-487d-a480-a642dad64ec4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xgznh\" (UID: \"359f3d6e-a08a-487d-a480-a642dad64ec4\") " pod="openshift-insights/insights-runtime-extractor-xgznh" Jul 09 04:32:46.756421 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:46.756384 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:46.796281 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:46.796254 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/359f3d6e-a08a-487d-a480-a642dad64ec4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xgznh\" (UID: \"359f3d6e-a08a-487d-a480-a642dad64ec4\") " pod="openshift-insights/insights-runtime-extractor-xgznh" Jul 09 04:32:46.796717 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:46.796303 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/359f3d6e-a08a-487d-a480-a642dad64ec4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xgznh\" (UID: \"359f3d6e-a08a-487d-a480-a642dad64ec4\") " pod="openshift-insights/insights-runtime-extractor-xgznh" Jul 09 04:32:46.796717 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:46.796377 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzqp7\" (UniqueName: \"kubernetes.io/projected/359f3d6e-a08a-487d-a480-a642dad64ec4-kube-api-access-zzqp7\") pod \"insights-runtime-extractor-xgznh\" (UID: \"359f3d6e-a08a-487d-a480-a642dad64ec4\") " pod="openshift-insights/insights-runtime-extractor-xgznh" Jul 09 04:32:46.796717 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:46.796387 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 04:32:46.796717 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:46.796400 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 04:32:46.796717 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:46.796429 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1db69bb4-f741-480b-9608-03c97923a6c1-registry-tls\") pod \"image-registry-75979d6b76-prn9d\" (UID: \"1db69bb4-f741-480b-9608-03c97923a6c1\") " pod="openshift-image-registry/image-registry-75979d6b76-prn9d" Jul 09 04:32:46.796717 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:46.796464 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/359f3d6e-a08a-487d-a480-a642dad64ec4-kube-rbac-proxy-cm podName:359f3d6e-a08a-487d-a480-a642dad64ec4 nodeName:}" failed. No retries permitted until 2026-07-09 04:32:47.296446093 +0000 UTC m=+39.547969669 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/359f3d6e-a08a-487d-a480-a642dad64ec4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xgznh" (UID: "359f3d6e-a08a-487d-a480-a642dad64ec4") : configmap "kube-rbac-proxy" not found Jul 09 04:32:46.796717 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:46.796518 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/359f3d6e-a08a-487d-a480-a642dad64ec4-data-volume\") pod \"insights-runtime-extractor-xgznh\" (UID: \"359f3d6e-a08a-487d-a480-a642dad64ec4\") " pod="openshift-insights/insights-runtime-extractor-xgznh" Jul 09 04:32:46.796717 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:46.796525 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/359f3d6e-a08a-487d-a480-a642dad64ec4-insights-runtime-extractor-tls podName:359f3d6e-a08a-487d-a480-a642dad64ec4 nodeName:}" failed. No retries permitted until 2026-07-09 04:32:47.296511215 +0000 UTC m=+39.548034786 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/359f3d6e-a08a-487d-a480-a642dad64ec4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xgznh" (UID: "359f3d6e-a08a-487d-a480-a642dad64ec4") : secret "insights-runtime-extractor-tls" not found Jul 09 04:32:46.796717 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:46.796527 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 04:32:46.796717 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:46.796545 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75979d6b76-prn9d: secret "image-registry-tls" not found Jul 09 04:32:46.796717 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:46.796566 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/359f3d6e-a08a-487d-a480-a642dad64ec4-crio-socket\") pod \"insights-runtime-extractor-xgznh\" (UID: \"359f3d6e-a08a-487d-a480-a642dad64ec4\") " pod="openshift-insights/insights-runtime-extractor-xgznh" Jul 09 04:32:46.796717 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:46.796590 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1db69bb4-f741-480b-9608-03c97923a6c1-registry-tls podName:1db69bb4-f741-480b-9608-03c97923a6c1 nodeName:}" failed. No retries permitted until 2026-07-09 04:32:50.796573632 +0000 UTC m=+43.048097188 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1db69bb4-f741-480b-9608-03c97923a6c1-registry-tls") pod "image-registry-75979d6b76-prn9d" (UID: "1db69bb4-f741-480b-9608-03c97923a6c1") : secret "image-registry-tls" not found Jul 09 04:32:46.797100 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:46.796766 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/359f3d6e-a08a-487d-a480-a642dad64ec4-crio-socket\") pod \"insights-runtime-extractor-xgznh\" (UID: \"359f3d6e-a08a-487d-a480-a642dad64ec4\") " pod="openshift-insights/insights-runtime-extractor-xgznh" Jul 09 04:32:46.797100 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:46.796821 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/359f3d6e-a08a-487d-a480-a642dad64ec4-data-volume\") pod \"insights-runtime-extractor-xgznh\" (UID: \"359f3d6e-a08a-487d-a480-a642dad64ec4\") " pod="openshift-insights/insights-runtime-extractor-xgznh" Jul 09 04:32:46.805717 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:46.805691 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzqp7\" (UniqueName: \"kubernetes.io/projected/359f3d6e-a08a-487d-a480-a642dad64ec4-kube-api-access-zzqp7\") pod \"insights-runtime-extractor-xgznh\" (UID: \"359f3d6e-a08a-487d-a480-a642dad64ec4\") " pod="openshift-insights/insights-runtime-extractor-xgznh" Jul 09 04:32:46.809054 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:46.809030 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:46.898024 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:46.897941 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766-metrics-tls\") pod \"dns-default-ct9t6\" (UID: \"7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766\") " pod="openshift-dns/dns-default-ct9t6" Jul 09 04:32:46.898179 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:46.898116 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 04:32:46.898301 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:46.898210 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766-metrics-tls podName:7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766 nodeName:}" failed. No retries permitted until 2026-07-09 04:32:50.898187911 +0000 UTC m=+43.149711484 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766-metrics-tls") pod "dns-default-ct9t6" (UID: "7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766") : secret "dns-default-metrics-tls" not found Jul 09 04:32:47.021707 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:47.021671 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:47.121143 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:47.120983 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:47.300790 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:47.300748 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/359f3d6e-a08a-487d-a480-a642dad64ec4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xgznh\" (UID: \"359f3d6e-a08a-487d-a480-a642dad64ec4\") " pod="openshift-insights/insights-runtime-extractor-xgznh" Jul 09 04:32:47.300956 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:47.300796 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/359f3d6e-a08a-487d-a480-a642dad64ec4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xgznh\" (UID: \"359f3d6e-a08a-487d-a480-a642dad64ec4\") " pod="openshift-insights/insights-runtime-extractor-xgznh" Jul 09 04:32:47.300956 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:47.300897 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 04:32:47.300956 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:47.300940 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 04:32:47.301083 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:47.300961 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/359f3d6e-a08a-487d-a480-a642dad64ec4-insights-runtime-extractor-tls podName:359f3d6e-a08a-487d-a480-a642dad64ec4 nodeName:}" failed. No retries permitted until 2026-07-09 04:32:48.300945378 +0000 UTC m=+40.552468945 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/359f3d6e-a08a-487d-a480-a642dad64ec4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xgznh" (UID: "359f3d6e-a08a-487d-a480-a642dad64ec4") : secret "insights-runtime-extractor-tls" not found Jul 09 04:32:47.301083 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:47.300990 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/359f3d6e-a08a-487d-a480-a642dad64ec4-kube-rbac-proxy-cm podName:359f3d6e-a08a-487d-a480-a642dad64ec4 nodeName:}" failed. No retries permitted until 2026-07-09 04:32:48.300977205 +0000 UTC m=+40.552500762 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/359f3d6e-a08a-487d-a480-a642dad64ec4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xgznh" (UID: "359f3d6e-a08a-487d-a480-a642dad64ec4") : configmap "kube-rbac-proxy" not found Jul 09 04:32:47.510600 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:47.510509 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cmpgg" event={"ID":"af45d649-376b-4efd-ad4c-9b5b3c28ab4d","Type":"ContainerStarted","Data":"b4a0d2625f82c0728a10cc449d14ed540d9a5a7e4c78e26638ebb792a8601bf7"} Jul 09 04:32:47.511831 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:47.511808 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wbngq" event={"ID":"964ab4da-c25d-46bd-97df-ffb2cd5b820b","Type":"ContainerStarted","Data":"d337d93b878d4021ef4c4751936aa085ee4ab4c0cc990ba4c920bb47fe40d320"} Jul 09 04:32:47.532930 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:47.532883 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-cmpgg" podStartSLOduration=14.059620675 podStartE2EDuration="39.532867381s" totalImagesPullingTime="25.473246706s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:32:08 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:32:47.531358338 +0000 UTC m=+39.782881907" watchObservedRunningTime="2026-07-09 04:32:47.532867381 +0000 UTC m=+39.784390969" Jul 09 04:32:47.544700 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:47.544654 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-wbngq" podStartSLOduration=2.018548952 podStartE2EDuration="4.544638248s" totalImagesPullingTime="2.526089296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:32:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:32:47.544596637 +0000 UTC m=+39.796120218" watchObservedRunningTime="2026-07-09 04:32:47.544638248 +0000 UTC m=+39.796161828" Jul 09 04:32:47.658823 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:47.658791 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:47.715554 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:47.715512 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:48.309542 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:48.309507 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/359f3d6e-a08a-487d-a480-a642dad64ec4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xgznh\" (UID: \"359f3d6e-a08a-487d-a480-a642dad64ec4\") " pod="openshift-insights/insights-runtime-extractor-xgznh" Jul 09 04:32:48.309542 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:48.309546 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/359f3d6e-a08a-487d-a480-a642dad64ec4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xgznh\" (UID: \"359f3d6e-a08a-487d-a480-a642dad64ec4\") " pod="openshift-insights/insights-runtime-extractor-xgznh" Jul 09 04:32:48.309989 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:48.309647 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 04:32:48.309989 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:48.309667 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 04:32:48.309989 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:48.309699 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/359f3d6e-a08a-487d-a480-a642dad64ec4-kube-rbac-proxy-cm podName:359f3d6e-a08a-487d-a480-a642dad64ec4 nodeName:}" failed. No retries permitted until 2026-07-09 04:32:50.30968522 +0000 UTC m=+42.561208775 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/359f3d6e-a08a-487d-a480-a642dad64ec4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xgznh" (UID: "359f3d6e-a08a-487d-a480-a642dad64ec4") : configmap "kube-rbac-proxy" not found Jul 09 04:32:48.309989 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:48.309719 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/359f3d6e-a08a-487d-a480-a642dad64ec4-insights-runtime-extractor-tls podName:359f3d6e-a08a-487d-a480-a642dad64ec4 nodeName:}" failed. No retries permitted until 2026-07-09 04:32:50.309704424 +0000 UTC m=+42.561228001 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/359f3d6e-a08a-487d-a480-a642dad64ec4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xgznh" (UID: "359f3d6e-a08a-487d-a480-a642dad64ec4") : secret "insights-runtime-extractor-tls" not found Jul 09 04:32:48.666260 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:48.666225 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:48.725238 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:48.725211 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:48.737804 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:48.737784 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xzgbm_7b2795d7-ba4b-46ea-bd2c-6b9d061e9746/dns-node-resolver/0.log" Jul 09 04:32:48.828901 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:48.828872 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:48.926622 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:48.926553 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:49.338848 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:49.338777 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-j7zhv_227fe7b3-97ee-435b-9e07-15cf67dd2beb/node-ca/0.log" Jul 09 04:32:49.562592 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:49.562564 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:50.326993 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:50.326950 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/359f3d6e-a08a-487d-a480-a642dad64ec4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xgznh\" (UID: \"359f3d6e-a08a-487d-a480-a642dad64ec4\") " pod="openshift-insights/insights-runtime-extractor-xgznh" Jul 09 04:32:50.326993 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:50.327002 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/359f3d6e-a08a-487d-a480-a642dad64ec4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xgznh\" (UID: \"359f3d6e-a08a-487d-a480-a642dad64ec4\") " pod="openshift-insights/insights-runtime-extractor-xgznh" Jul 09 04:32:50.327203 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:50.327104 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 04:32:50.327203 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:50.327174 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/359f3d6e-a08a-487d-a480-a642dad64ec4-insights-runtime-extractor-tls podName:359f3d6e-a08a-487d-a480-a642dad64ec4 nodeName:}" failed. No retries permitted until 2026-07-09 04:32:54.327156954 +0000 UTC m=+46.578680510 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/359f3d6e-a08a-487d-a480-a642dad64ec4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xgznh" (UID: "359f3d6e-a08a-487d-a480-a642dad64ec4") : secret "insights-runtime-extractor-tls" not found Jul 09 04:32:50.327203 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:50.327105 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 04:32:50.327302 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:50.327228 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/359f3d6e-a08a-487d-a480-a642dad64ec4-kube-rbac-proxy-cm podName:359f3d6e-a08a-487d-a480-a642dad64ec4 nodeName:}" failed. No retries permitted until 2026-07-09 04:32:54.327216522 +0000 UTC m=+46.578740079 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/359f3d6e-a08a-487d-a480-a642dad64ec4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xgznh" (UID: "359f3d6e-a08a-487d-a480-a642dad64ec4") : configmap "kube-rbac-proxy" not found Jul 09 04:32:50.538142 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:50.538112 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wbngq_964ab4da-c25d-46bd-97df-ffb2cd5b820b/serve-healthcheck-canary/0.log" Jul 09 04:32:50.831558 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:50.831514 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1db69bb4-f741-480b-9608-03c97923a6c1-registry-tls\") pod \"image-registry-75979d6b76-prn9d\" (UID: \"1db69bb4-f741-480b-9608-03c97923a6c1\") " pod="openshift-image-registry/image-registry-75979d6b76-prn9d" Jul 09 04:32:50.831710 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:50.831664 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 04:32:50.831710 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:50.831685 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75979d6b76-prn9d: secret "image-registry-tls" not found Jul 09 04:32:50.831789 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:50.831739 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1db69bb4-f741-480b-9608-03c97923a6c1-registry-tls podName:1db69bb4-f741-480b-9608-03c97923a6c1 nodeName:}" failed. No retries permitted until 2026-07-09 04:32:58.831720789 +0000 UTC m=+51.083244345 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1db69bb4-f741-480b-9608-03c97923a6c1-registry-tls") pod "image-registry-75979d6b76-prn9d" (UID: "1db69bb4-f741-480b-9608-03c97923a6c1") : secret "image-registry-tls" not found Jul 09 04:32:50.932475 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:50.932438 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766-metrics-tls\") pod \"dns-default-ct9t6\" (UID: \"7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766\") " pod="openshift-dns/dns-default-ct9t6" Jul 09 04:32:50.932633 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:50.932592 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 04:32:50.932669 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:50.932650 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766-metrics-tls podName:7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766 nodeName:}" failed. No retries permitted until 2026-07-09 04:32:58.932633918 +0000 UTC m=+51.184157491 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766-metrics-tls") pod "dns-default-ct9t6" (UID: "7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766") : secret "dns-default-metrics-tls" not found Jul 09 04:32:52.765212 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:52.765184 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:54.361086 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:54.361041 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/359f3d6e-a08a-487d-a480-a642dad64ec4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xgznh\" (UID: \"359f3d6e-a08a-487d-a480-a642dad64ec4\") " pod="openshift-insights/insights-runtime-extractor-xgznh" Jul 09 04:32:54.361707 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:54.361093 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/359f3d6e-a08a-487d-a480-a642dad64ec4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xgznh\" (UID: \"359f3d6e-a08a-487d-a480-a642dad64ec4\") " pod="openshift-insights/insights-runtime-extractor-xgznh" Jul 09 04:32:54.361707 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:54.361187 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 04:32:54.361707 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:54.361218 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 04:32:54.361707 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:54.361251 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/359f3d6e-a08a-487d-a480-a642dad64ec4-kube-rbac-proxy-cm podName:359f3d6e-a08a-487d-a480-a642dad64ec4 nodeName:}" failed. No retries permitted until 2026-07-09 04:33:02.361233996 +0000 UTC m=+54.612757570 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/359f3d6e-a08a-487d-a480-a642dad64ec4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xgznh" (UID: "359f3d6e-a08a-487d-a480-a642dad64ec4") : configmap "kube-rbac-proxy" not found Jul 09 04:32:54.361707 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:54.361279 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/359f3d6e-a08a-487d-a480-a642dad64ec4-insights-runtime-extractor-tls podName:359f3d6e-a08a-487d-a480-a642dad64ec4 nodeName:}" failed. No retries permitted until 2026-07-09 04:33:02.361262566 +0000 UTC m=+54.612786121 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/359f3d6e-a08a-487d-a480-a642dad64ec4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xgznh" (UID: "359f3d6e-a08a-487d-a480-a642dad64ec4") : secret "insights-runtime-extractor-tls" not found Jul 09 04:32:55.517410 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:55.517371 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:55.936822 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:55.936792 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:56.113103 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:56.113071 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:56.213609 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:56.213538 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:56.315189 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:56.315156 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:56.413985 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:56.413955 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:57.019255 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:57.019227 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:57.428701 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:57.428662 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:57.621801 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:57.621769 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:58.224801 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:58.224760 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:58.624254 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:58.624178 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:58.898320 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:58.898235 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1db69bb4-f741-480b-9608-03c97923a6c1-registry-tls\") pod \"image-registry-75979d6b76-prn9d\" (UID: \"1db69bb4-f741-480b-9608-03c97923a6c1\") " pod="openshift-image-registry/image-registry-75979d6b76-prn9d" Jul 09 04:32:58.898480 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:58.898422 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 04:32:58.898480 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:58.898442 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75979d6b76-prn9d: secret "image-registry-tls" not found Jul 09 04:32:58.898551 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:58.898508 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1db69bb4-f741-480b-9608-03c97923a6c1-registry-tls podName:1db69bb4-f741-480b-9608-03c97923a6c1 nodeName:}" failed. No retries permitted until 2026-07-09 04:33:14.898490101 +0000 UTC m=+67.150013676 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1db69bb4-f741-480b-9608-03c97923a6c1-registry-tls") pod "image-registry-75979d6b76-prn9d" (UID: "1db69bb4-f741-480b-9608-03c97923a6c1") : secret "image-registry-tls" not found Jul 09 04:32:58.999018 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:58.998985 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766-metrics-tls\") pod \"dns-default-ct9t6\" (UID: \"7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766\") " pod="openshift-dns/dns-default-ct9t6" Jul 09 04:32:58.999166 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:58.999127 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 04:32:58.999236 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:32:58.999224 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766-metrics-tls podName:7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766 nodeName:}" failed. No retries permitted until 2026-07-09 04:33:14.9991949 +0000 UTC m=+67.250718456 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766-metrics-tls") pod "dns-default-ct9t6" (UID: "7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766") : secret "dns-default-metrics-tls" not found Jul 09 04:32:59.021081 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:59.021045 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:59.242005 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:59.241975 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:59.616789 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:59.616712 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:59.721157 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:59.721127 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:59.824054 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:59.824018 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:32:59.925547 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:32:59.925526 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:00.225196 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:00.225106 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:00.606322 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:00.606250 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:00.688778 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:00.688752 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:00.832469 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:00.832438 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:01.232307 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:01.232279 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:01.334071 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:01.334038 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:01.436993 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:01.436965 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:01.823596 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:01.823564 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:02.286731 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:02.286701 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:02.345809 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:02.345776 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:02.425008 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:02.424977 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/359f3d6e-a08a-487d-a480-a642dad64ec4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xgznh\" (UID: \"359f3d6e-a08a-487d-a480-a642dad64ec4\") " pod="openshift-insights/insights-runtime-extractor-xgznh" Jul 09 04:33:02.425101 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:02.425016 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/359f3d6e-a08a-487d-a480-a642dad64ec4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xgznh\" (UID: \"359f3d6e-a08a-487d-a480-a642dad64ec4\") " pod="openshift-insights/insights-runtime-extractor-xgznh" Jul 09 04:33:02.425145 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:33:02.425127 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 04:33:02.425210 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:33:02.425199 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/359f3d6e-a08a-487d-a480-a642dad64ec4-insights-runtime-extractor-tls podName:359f3d6e-a08a-487d-a480-a642dad64ec4 nodeName:}" failed. No retries permitted until 2026-07-09 04:33:18.425180998 +0000 UTC m=+70.676704569 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/359f3d6e-a08a-487d-a480-a642dad64ec4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xgznh" (UID: "359f3d6e-a08a-487d-a480-a642dad64ec4") : secret "insights-runtime-extractor-tls" not found Jul 09 04:33:02.425673 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:02.425655 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/359f3d6e-a08a-487d-a480-a642dad64ec4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xgznh\" (UID: \"359f3d6e-a08a-487d-a480-a642dad64ec4\") " pod="openshift-insights/insights-runtime-extractor-xgznh" Jul 09 04:33:02.630177 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:02.630106 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:02.823975 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:02.823944 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:03.417067 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:03.417036 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:03.819227 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:03.819141 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:04.225071 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:04.225038 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:04.424104 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:04.424073 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:04.817972 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:04.817943 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:05.341778 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:05.341748 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:05.400740 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:05.400709 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:05.453532 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:05.453472 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:05.821631 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:05.821590 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:06.701164 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:06.701136 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:06.802263 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:06.802234 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:08.494141 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:08.494116 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t85dg" Jul 09 04:33:11.282617 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.282577 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-596786469b-l988v"] Jul 09 04:33:11.286026 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.286003 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-596786469b-l988v" Jul 09 04:33:11.289658 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.289631 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 09 04:33:11.289658 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.289645 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 09 04:33:11.289847 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.289709 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 09 04:33:11.289847 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.289645 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 09 04:33:11.289847 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.289735 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 09 04:33:11.289847 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.289828 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 09 04:33:11.290992 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.290251 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 09 04:33:11.296524 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.296500 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-596786469b-l988v"] Jul 09 04:33:11.363294 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.363262 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-75979d6b76-prn9d"] Jul 09 04:33:11.363508 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:33:11.363489 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-75979d6b76-prn9d" podUID="1db69bb4-f741-480b-9608-03c97923a6c1" Jul 09 04:33:11.375030 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.374990 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:11.395631 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.395603 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/0bc1140e-97ff-4c83-a076-127486b1b83c-ca\") pod \"cluster-proxy-proxy-agent-596786469b-l988v\" (UID: \"0bc1140e-97ff-4c83-a076-127486b1b83c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-596786469b-l988v" Jul 09 04:33:11.395631 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.395632 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/0bc1140e-97ff-4c83-a076-127486b1b83c-hub\") pod \"cluster-proxy-proxy-agent-596786469b-l988v\" (UID: \"0bc1140e-97ff-4c83-a076-127486b1b83c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-596786469b-l988v" Jul 09 04:33:11.395770 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.395659 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gs8kd\" (UniqueName: \"kubernetes.io/projected/0bc1140e-97ff-4c83-a076-127486b1b83c-kube-api-access-gs8kd\") pod \"cluster-proxy-proxy-agent-596786469b-l988v\" (UID: \"0bc1140e-97ff-4c83-a076-127486b1b83c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-596786469b-l988v" Jul 09 04:33:11.395770 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.395718 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0bc1140e-97ff-4c83-a076-127486b1b83c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-596786469b-l988v\" (UID: \"0bc1140e-97ff-4c83-a076-127486b1b83c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-596786469b-l988v" Jul 09 04:33:11.395770 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.395743 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/0bc1140e-97ff-4c83-a076-127486b1b83c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-596786469b-l988v\" (UID: \"0bc1140e-97ff-4c83-a076-127486b1b83c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-596786469b-l988v" Jul 09 04:33:11.395915 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.395874 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bc1140e-97ff-4c83-a076-127486b1b83c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-596786469b-l988v\" (UID: \"0bc1140e-97ff-4c83-a076-127486b1b83c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-596786469b-l988v" Jul 09 04:33:11.409983 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.409958 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-76b76c5bc7-4r6xd"] Jul 09 04:33:11.411679 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.411666 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76b76c5bc7-4r6xd" Jul 09 04:33:11.426708 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.426680 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76b76c5bc7-4r6xd"] Jul 09 04:33:11.496556 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.496514 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bc1140e-97ff-4c83-a076-127486b1b83c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-596786469b-l988v\" (UID: \"0bc1140e-97ff-4c83-a076-127486b1b83c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-596786469b-l988v" Jul 09 04:33:11.496752 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.496566 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/974fb6f3-fe6f-4ea7-9987-1813f7f037a1-registry-tls\") pod \"image-registry-76b76c5bc7-4r6xd\" (UID: \"974fb6f3-fe6f-4ea7-9987-1813f7f037a1\") " pod="openshift-image-registry/image-registry-76b76c5bc7-4r6xd" Jul 09 04:33:11.496752 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.496597 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs8kd\" (UniqueName: \"kubernetes.io/projected/0bc1140e-97ff-4c83-a076-127486b1b83c-kube-api-access-gs8kd\") pod \"cluster-proxy-proxy-agent-596786469b-l988v\" (UID: \"0bc1140e-97ff-4c83-a076-127486b1b83c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-596786469b-l988v" Jul 09 04:33:11.496752 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.496615 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/974fb6f3-fe6f-4ea7-9987-1813f7f037a1-registry-certificates\") pod \"image-registry-76b76c5bc7-4r6xd\" (UID: \"974fb6f3-fe6f-4ea7-9987-1813f7f037a1\") " pod="openshift-image-registry/image-registry-76b76c5bc7-4r6xd" Jul 09 04:33:11.496752 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.496630 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/974fb6f3-fe6f-4ea7-9987-1813f7f037a1-bound-sa-token\") pod \"image-registry-76b76c5bc7-4r6xd\" (UID: \"974fb6f3-fe6f-4ea7-9987-1813f7f037a1\") " pod="openshift-image-registry/image-registry-76b76c5bc7-4r6xd" Jul 09 04:33:11.496752 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.496664 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/974fb6f3-fe6f-4ea7-9987-1813f7f037a1-image-registry-private-configuration\") pod \"image-registry-76b76c5bc7-4r6xd\" (UID: \"974fb6f3-fe6f-4ea7-9987-1813f7f037a1\") " pod="openshift-image-registry/image-registry-76b76c5bc7-4r6xd" Jul 09 04:33:11.496752 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.496689 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rk5x7\" (UniqueName: \"kubernetes.io/projected/974fb6f3-fe6f-4ea7-9987-1813f7f037a1-kube-api-access-rk5x7\") pod \"image-registry-76b76c5bc7-4r6xd\" (UID: \"974fb6f3-fe6f-4ea7-9987-1813f7f037a1\") " pod="openshift-image-registry/image-registry-76b76c5bc7-4r6xd" Jul 09 04:33:11.496976 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.496882 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/974fb6f3-fe6f-4ea7-9987-1813f7f037a1-installation-pull-secrets\") pod \"image-registry-76b76c5bc7-4r6xd\" (UID: \"974fb6f3-fe6f-4ea7-9987-1813f7f037a1\") " pod="openshift-image-registry/image-registry-76b76c5bc7-4r6xd" Jul 09 04:33:11.496976 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.496917 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/0bc1140e-97ff-4c83-a076-127486b1b83c-ca\") pod \"cluster-proxy-proxy-agent-596786469b-l988v\" (UID: \"0bc1140e-97ff-4c83-a076-127486b1b83c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-596786469b-l988v" Jul 09 04:33:11.496976 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.496933 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0bc1140e-97ff-4c83-a076-127486b1b83c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-596786469b-l988v\" (UID: \"0bc1140e-97ff-4c83-a076-127486b1b83c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-596786469b-l988v" Jul 09 04:33:11.497104 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.496976 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/974fb6f3-fe6f-4ea7-9987-1813f7f037a1-trusted-ca\") pod \"image-registry-76b76c5bc7-4r6xd\" (UID: \"974fb6f3-fe6f-4ea7-9987-1813f7f037a1\") " pod="openshift-image-registry/image-registry-76b76c5bc7-4r6xd" Jul 09 04:33:11.497104 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.497059 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/0bc1140e-97ff-4c83-a076-127486b1b83c-hub\") pod \"cluster-proxy-proxy-agent-596786469b-l988v\" (UID: \"0bc1140e-97ff-4c83-a076-127486b1b83c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-596786469b-l988v" Jul 09 04:33:11.497104 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.497086 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/0bc1140e-97ff-4c83-a076-127486b1b83c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-596786469b-l988v\" (UID: \"0bc1140e-97ff-4c83-a076-127486b1b83c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-596786469b-l988v" Jul 09 04:33:11.497246 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.497115 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/974fb6f3-fe6f-4ea7-9987-1813f7f037a1-ca-trust-extracted\") pod \"image-registry-76b76c5bc7-4r6xd\" (UID: \"974fb6f3-fe6f-4ea7-9987-1813f7f037a1\") " pod="openshift-image-registry/image-registry-76b76c5bc7-4r6xd" Jul 09 04:33:11.497387 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.497362 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bc1140e-97ff-4c83-a076-127486b1b83c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-596786469b-l988v\" (UID: \"0bc1140e-97ff-4c83-a076-127486b1b83c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-596786469b-l988v" Jul 09 04:33:11.498452 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.498424 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:11.500300 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.500267 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/0bc1140e-97ff-4c83-a076-127486b1b83c-ca\") pod \"cluster-proxy-proxy-agent-596786469b-l988v\" (UID: \"0bc1140e-97ff-4c83-a076-127486b1b83c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-596786469b-l988v" Jul 09 04:33:11.500440 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.500421 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/0bc1140e-97ff-4c83-a076-127486b1b83c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-596786469b-l988v\" (UID: \"0bc1140e-97ff-4c83-a076-127486b1b83c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-596786469b-l988v" Jul 09 04:33:11.500440 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.500429 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0bc1140e-97ff-4c83-a076-127486b1b83c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-596786469b-l988v\" (UID: \"0bc1140e-97ff-4c83-a076-127486b1b83c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-596786469b-l988v" Jul 09 04:33:11.500552 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.500503 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/0bc1140e-97ff-4c83-a076-127486b1b83c-hub\") pod \"cluster-proxy-proxy-agent-596786469b-l988v\" (UID: \"0bc1140e-97ff-4c83-a076-127486b1b83c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-596786469b-l988v" Jul 09 04:33:11.507495 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.507467 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gs8kd\" (UniqueName: \"kubernetes.io/projected/0bc1140e-97ff-4c83-a076-127486b1b83c-kube-api-access-gs8kd\") pod \"cluster-proxy-proxy-agent-596786469b-l988v\" (UID: \"0bc1140e-97ff-4c83-a076-127486b1b83c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-596786469b-l988v" Jul 09 04:33:11.564756 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.564681 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75979d6b76-prn9d" Jul 09 04:33:11.569910 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.569892 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75979d6b76-prn9d" Jul 09 04:33:11.589922 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.589893 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:11.598074 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.598054 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1db69bb4-f741-480b-9608-03c97923a6c1-registry-certificates\") pod \"1db69bb4-f741-480b-9608-03c97923a6c1\" (UID: \"1db69bb4-f741-480b-9608-03c97923a6c1\") " Jul 09 04:33:11.598141 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.598087 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1db69bb4-f741-480b-9608-03c97923a6c1-image-registry-private-configuration\") pod \"1db69bb4-f741-480b-9608-03c97923a6c1\" (UID: \"1db69bb4-f741-480b-9608-03c97923a6c1\") " Jul 09 04:33:11.598141 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.598111 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9wvhp\" (UniqueName: \"kubernetes.io/projected/1db69bb4-f741-480b-9608-03c97923a6c1-kube-api-access-9wvhp\") pod \"1db69bb4-f741-480b-9608-03c97923a6c1\" (UID: \"1db69bb4-f741-480b-9608-03c97923a6c1\") " Jul 09 04:33:11.598206 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.598146 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1db69bb4-f741-480b-9608-03c97923a6c1-installation-pull-secrets\") pod \"1db69bb4-f741-480b-9608-03c97923a6c1\" (UID: \"1db69bb4-f741-480b-9608-03c97923a6c1\") " Jul 09 04:33:11.598206 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.598188 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1db69bb4-f741-480b-9608-03c97923a6c1-bound-sa-token\") pod \"1db69bb4-f741-480b-9608-03c97923a6c1\" (UID: \"1db69bb4-f741-480b-9608-03c97923a6c1\") " Jul 09 04:33:11.598278 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.598221 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1db69bb4-f741-480b-9608-03c97923a6c1-ca-trust-extracted\") pod \"1db69bb4-f741-480b-9608-03c97923a6c1\" (UID: \"1db69bb4-f741-480b-9608-03c97923a6c1\") " Jul 09 04:33:11.598278 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.598249 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1db69bb4-f741-480b-9608-03c97923a6c1-trusted-ca\") pod \"1db69bb4-f741-480b-9608-03c97923a6c1\" (UID: \"1db69bb4-f741-480b-9608-03c97923a6c1\") " Jul 09 04:33:11.598457 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.598417 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/974fb6f3-fe6f-4ea7-9987-1813f7f037a1-installation-pull-secrets\") pod \"image-registry-76b76c5bc7-4r6xd\" (UID: \"974fb6f3-fe6f-4ea7-9987-1813f7f037a1\") " pod="openshift-image-registry/image-registry-76b76c5bc7-4r6xd" Jul 09 04:33:11.598507 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.598456 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/974fb6f3-fe6f-4ea7-9987-1813f7f037a1-trusted-ca\") pod \"image-registry-76b76c5bc7-4r6xd\" (UID: \"974fb6f3-fe6f-4ea7-9987-1813f7f037a1\") " pod="openshift-image-registry/image-registry-76b76c5bc7-4r6xd" Jul 09 04:33:11.598507 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.598465 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1db69bb4-f741-480b-9608-03c97923a6c1-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "1db69bb4-f741-480b-9608-03c97923a6c1" (UID: "1db69bb4-f741-480b-9608-03c97923a6c1"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:33:11.598507 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.598502 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/974fb6f3-fe6f-4ea7-9987-1813f7f037a1-ca-trust-extracted\") pod \"image-registry-76b76c5bc7-4r6xd\" (UID: \"974fb6f3-fe6f-4ea7-9987-1813f7f037a1\") " pod="openshift-image-registry/image-registry-76b76c5bc7-4r6xd" Jul 09 04:33:11.598655 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.598585 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/974fb6f3-fe6f-4ea7-9987-1813f7f037a1-registry-tls\") pod \"image-registry-76b76c5bc7-4r6xd\" (UID: \"974fb6f3-fe6f-4ea7-9987-1813f7f037a1\") " pod="openshift-image-registry/image-registry-76b76c5bc7-4r6xd" Jul 09 04:33:11.598655 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.598625 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/974fb6f3-fe6f-4ea7-9987-1813f7f037a1-registry-certificates\") pod \"image-registry-76b76c5bc7-4r6xd\" (UID: \"974fb6f3-fe6f-4ea7-9987-1813f7f037a1\") " pod="openshift-image-registry/image-registry-76b76c5bc7-4r6xd" Jul 09 04:33:11.598758 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.598659 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/974fb6f3-fe6f-4ea7-9987-1813f7f037a1-bound-sa-token\") pod \"image-registry-76b76c5bc7-4r6xd\" (UID: \"974fb6f3-fe6f-4ea7-9987-1813f7f037a1\") " pod="openshift-image-registry/image-registry-76b76c5bc7-4r6xd" Jul 09 04:33:11.598758 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.598697 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/974fb6f3-fe6f-4ea7-9987-1813f7f037a1-image-registry-private-configuration\") pod \"image-registry-76b76c5bc7-4r6xd\" (UID: \"974fb6f3-fe6f-4ea7-9987-1813f7f037a1\") " pod="openshift-image-registry/image-registry-76b76c5bc7-4r6xd" Jul 09 04:33:11.598758 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.598718 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk5x7\" (UniqueName: \"kubernetes.io/projected/974fb6f3-fe6f-4ea7-9987-1813f7f037a1-kube-api-access-rk5x7\") pod \"image-registry-76b76c5bc7-4r6xd\" (UID: \"974fb6f3-fe6f-4ea7-9987-1813f7f037a1\") " pod="openshift-image-registry/image-registry-76b76c5bc7-4r6xd" Jul 09 04:33:11.598898 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.598793 2565 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1db69bb4-f741-480b-9608-03c97923a6c1-registry-certificates\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:33:11.598898 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.598812 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1db69bb4-f741-480b-9608-03c97923a6c1-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "1db69bb4-f741-480b-9608-03c97923a6c1" (UID: "1db69bb4-f741-480b-9608-03c97923a6c1"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:33:11.599094 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.599070 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/974fb6f3-fe6f-4ea7-9987-1813f7f037a1-ca-trust-extracted\") pod \"image-registry-76b76c5bc7-4r6xd\" (UID: \"974fb6f3-fe6f-4ea7-9987-1813f7f037a1\") " pod="openshift-image-registry/image-registry-76b76c5bc7-4r6xd" Jul 09 04:33:11.599597 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.599569 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1db69bb4-f741-480b-9608-03c97923a6c1-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "1db69bb4-f741-480b-9608-03c97923a6c1" (UID: "1db69bb4-f741-480b-9608-03c97923a6c1"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:33:11.600398 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.600331 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/974fb6f3-fe6f-4ea7-9987-1813f7f037a1-trusted-ca\") pod \"image-registry-76b76c5bc7-4r6xd\" (UID: \"974fb6f3-fe6f-4ea7-9987-1813f7f037a1\") " pod="openshift-image-registry/image-registry-76b76c5bc7-4r6xd" Jul 09 04:33:11.600939 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.600825 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1db69bb4-f741-480b-9608-03c97923a6c1-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "1db69bb4-f741-480b-9608-03c97923a6c1" (UID: "1db69bb4-f741-480b-9608-03c97923a6c1"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:33:11.600939 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.600844 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1db69bb4-f741-480b-9608-03c97923a6c1-kube-api-access-9wvhp" (OuterVolumeSpecName: "kube-api-access-9wvhp") pod "1db69bb4-f741-480b-9608-03c97923a6c1" (UID: "1db69bb4-f741-480b-9608-03c97923a6c1"). InnerVolumeSpecName "kube-api-access-9wvhp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:33:11.600939 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.600870 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1db69bb4-f741-480b-9608-03c97923a6c1-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "1db69bb4-f741-480b-9608-03c97923a6c1" (UID: "1db69bb4-f741-480b-9608-03c97923a6c1"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:33:11.601166 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.601042 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/974fb6f3-fe6f-4ea7-9987-1813f7f037a1-registry-certificates\") pod \"image-registry-76b76c5bc7-4r6xd\" (UID: \"974fb6f3-fe6f-4ea7-9987-1813f7f037a1\") " pod="openshift-image-registry/image-registry-76b76c5bc7-4r6xd" Jul 09 04:33:11.601353 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.601310 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1db69bb4-f741-480b-9608-03c97923a6c1-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "1db69bb4-f741-480b-9608-03c97923a6c1" (UID: "1db69bb4-f741-480b-9608-03c97923a6c1"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:33:11.602488 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.602471 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/974fb6f3-fe6f-4ea7-9987-1813f7f037a1-registry-tls\") pod \"image-registry-76b76c5bc7-4r6xd\" (UID: \"974fb6f3-fe6f-4ea7-9987-1813f7f037a1\") " pod="openshift-image-registry/image-registry-76b76c5bc7-4r6xd" Jul 09 04:33:11.603024 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.602996 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/974fb6f3-fe6f-4ea7-9987-1813f7f037a1-installation-pull-secrets\") pod \"image-registry-76b76c5bc7-4r6xd\" (UID: \"974fb6f3-fe6f-4ea7-9987-1813f7f037a1\") " pod="openshift-image-registry/image-registry-76b76c5bc7-4r6xd" Jul 09 04:33:11.603267 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.603127 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/974fb6f3-fe6f-4ea7-9987-1813f7f037a1-image-registry-private-configuration\") pod \"image-registry-76b76c5bc7-4r6xd\" (UID: \"974fb6f3-fe6f-4ea7-9987-1813f7f037a1\") " pod="openshift-image-registry/image-registry-76b76c5bc7-4r6xd" Jul 09 04:33:11.606245 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.606221 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-596786469b-l988v" Jul 09 04:33:11.606639 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.606619 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/974fb6f3-fe6f-4ea7-9987-1813f7f037a1-bound-sa-token\") pod \"image-registry-76b76c5bc7-4r6xd\" (UID: \"974fb6f3-fe6f-4ea7-9987-1813f7f037a1\") " pod="openshift-image-registry/image-registry-76b76c5bc7-4r6xd" Jul 09 04:33:11.606682 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.606644 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rk5x7\" (UniqueName: \"kubernetes.io/projected/974fb6f3-fe6f-4ea7-9987-1813f7f037a1-kube-api-access-rk5x7\") pod \"image-registry-76b76c5bc7-4r6xd\" (UID: \"974fb6f3-fe6f-4ea7-9987-1813f7f037a1\") " pod="openshift-image-registry/image-registry-76b76c5bc7-4r6xd" Jul 09 04:33:11.699667 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.699639 2565 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1db69bb4-f741-480b-9608-03c97923a6c1-ca-trust-extracted\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:33:11.699667 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.699666 2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1db69bb4-f741-480b-9608-03c97923a6c1-trusted-ca\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:33:11.699874 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.699677 2565 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1db69bb4-f741-480b-9608-03c97923a6c1-image-registry-private-configuration\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:33:11.699874 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.699686 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9wvhp\" (UniqueName: \"kubernetes.io/projected/1db69bb4-f741-480b-9608-03c97923a6c1-kube-api-access-9wvhp\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:33:11.699874 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.699694 2565 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1db69bb4-f741-480b-9608-03c97923a6c1-installation-pull-secrets\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:33:11.699874 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.699702 2565 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1db69bb4-f741-480b-9608-03c97923a6c1-bound-sa-token\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:33:11.722936 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.722904 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8hcw8\"" Jul 09 04:33:11.726524 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.726497 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-596786469b-l988v"] Jul 09 04:33:11.729540 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:33:11.729514 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0bc1140e_97ff_4c83_a076_127486b1b83c.slice/crio-e76d93e74575ba93474008b2f0e0c812ea2bbcfa95bebbfaf2ba1e476dacdc63 WatchSource:0}: Error finding container e76d93e74575ba93474008b2f0e0c812ea2bbcfa95bebbfaf2ba1e476dacdc63: Status 404 returned error can't find the container with id e76d93e74575ba93474008b2f0e0c812ea2bbcfa95bebbfaf2ba1e476dacdc63 Jul 09 04:33:11.730616 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.730599 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76b76c5bc7-4r6xd" Jul 09 04:33:11.850291 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:11.850211 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76b76c5bc7-4r6xd"] Jul 09 04:33:11.853210 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:33:11.853186 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod974fb6f3_fe6f_4ea7_9987_1813f7f037a1.slice/crio-31357419cb6721062a330255a3e72255ec0eec92edd22af741743fc4ee1bb42a WatchSource:0}: Error finding container 31357419cb6721062a330255a3e72255ec0eec92edd22af741743fc4ee1bb42a: Status 404 returned error can't find the container with id 31357419cb6721062a330255a3e72255ec0eec92edd22af741743fc4ee1bb42a Jul 09 04:33:12.569857 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:12.569764 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76b76c5bc7-4r6xd" event={"ID":"974fb6f3-fe6f-4ea7-9987-1813f7f037a1","Type":"ContainerStarted","Data":"480638cdfaa5dbb42600b9d793aaa46963810611ced0fa9b8c51088f3ec4fbed"} Jul 09 04:33:12.569857 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:12.569819 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76b76c5bc7-4r6xd" event={"ID":"974fb6f3-fe6f-4ea7-9987-1813f7f037a1","Type":"ContainerStarted","Data":"31357419cb6721062a330255a3e72255ec0eec92edd22af741743fc4ee1bb42a"} Jul 09 04:33:12.570586 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:12.569867 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-76b76c5bc7-4r6xd" Jul 09 04:33:12.571882 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:12.571851 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-596786469b-l988v" event={"ID":"0bc1140e-97ff-4c83-a076-127486b1b83c","Type":"ContainerStarted","Data":"e76d93e74575ba93474008b2f0e0c812ea2bbcfa95bebbfaf2ba1e476dacdc63"} Jul 09 04:33:12.572050 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:12.572010 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75979d6b76-prn9d" Jul 09 04:33:12.593770 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:12.593447 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-76b76c5bc7-4r6xd" podStartSLOduration=1.593428264 podStartE2EDuration="1.593428264s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:33:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:33:12.592081014 +0000 UTC m=+64.843604593" watchObservedRunningTime="2026-07-09 04:33:12.593428264 +0000 UTC m=+64.844951842" Jul 09 04:33:12.617909 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:12.617867 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-75979d6b76-prn9d"] Jul 09 04:33:12.629751 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:12.629717 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-75979d6b76-prn9d"] Jul 09 04:33:12.707103 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:12.707047 2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1db69bb4-f741-480b-9608-03c97923a6c1-registry-tls\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:33:12.821921 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:12.821798 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:12.917676 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:12.917142 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:12.989648 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:12.989612 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:13.055537 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:13.055498 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:13.199161 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:13.199084 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:13.268279 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:13.268248 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:13.327459 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:13.327423 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:13.512582 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:13.512491 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:13.673667 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:13.673624 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:13.867582 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:13.867501 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:13.917538 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:13.917498 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa-metrics-certs\") pod \"network-metrics-daemon-gpx47\" (UID: \"dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa\") " pod="openshift-multus/network-metrics-daemon-gpx47" Jul 09 04:33:13.917729 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:13.917570 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0a12812-bbc1-4b88-a2f2-858d22d67e64-original-pull-secret\") pod \"global-pull-secret-syncer-zl4tp\" (UID: \"d0a12812-bbc1-4b88-a2f2-858d22d67e64\") " pod="kube-system/global-pull-secret-syncer-zl4tp" Jul 09 04:33:13.920003 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:13.919973 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 04:33:13.920678 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:13.920659 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 04:33:13.923224 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:13.923195 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:13.931857 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:13.931092 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa-metrics-certs\") pod \"network-metrics-daemon-gpx47\" (UID: \"dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa\") " pod="openshift-multus/network-metrics-daemon-gpx47" Jul 09 04:33:13.931857 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:13.931172 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0a12812-bbc1-4b88-a2f2-858d22d67e64-original-pull-secret\") pod \"global-pull-secret-syncer-zl4tp\" (UID: \"d0a12812-bbc1-4b88-a2f2-858d22d67e64\") " pod="kube-system/global-pull-secret-syncer-zl4tp" Jul 09 04:33:13.984385 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:13.984349 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:14.018182 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:14.018143 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-886q7\" (UniqueName: \"kubernetes.io/projected/f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59-kube-api-access-886q7\") pod \"network-check-target-bgsl7\" (UID: \"f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59\") " pod="openshift-network-diagnostics/network-check-target-bgsl7" Jul 09 04:33:14.020878 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:14.020850 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 04:33:14.031688 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:14.031659 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 04:33:14.042611 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:14.042582 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-886q7\" (UniqueName: \"kubernetes.io/projected/f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59-kube-api-access-886q7\") pod \"network-check-target-bgsl7\" (UID: \"f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59\") " pod="openshift-network-diagnostics/network-check-target-bgsl7" Jul 09 04:33:14.100638 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:14.100607 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zl4tp" Jul 09 04:33:14.109984 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:14.109941 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8mcbs\"" Jul 09 04:33:14.115073 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:14.115051 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bx88j\"" Jul 09 04:33:14.118082 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:14.118026 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bgsl7" Jul 09 04:33:14.124032 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:14.123995 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gpx47" Jul 09 04:33:14.199314 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:14.199073 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1db69bb4-f741-480b-9608-03c97923a6c1" path="/var/lib/kubelet/pods/1db69bb4-f741-480b-9608-03c97923a6c1/volumes" Jul 09 04:33:14.253575 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:14.253546 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zl4tp"] Jul 09 04:33:14.273838 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:14.273806 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bgsl7"] Jul 09 04:33:14.288726 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:14.288701 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gpx47"] Jul 09 04:33:14.291554 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:14.291529 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:14.486927 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:14.486895 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:14.548062 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:14.548031 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:14.867183 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:14.867109 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:15.028879 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:15.028845 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766-metrics-tls\") pod \"dns-default-ct9t6\" (UID: \"7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766\") " pod="openshift-dns/dns-default-ct9t6" Jul 09 04:33:15.031287 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:15.031261 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766-metrics-tls\") pod \"dns-default-ct9t6\" (UID: \"7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766\") " pod="openshift-dns/dns-default-ct9t6" Jul 09 04:33:15.077217 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:15.077183 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:15.222623 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:15.222595 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-58ffk\"" Jul 09 04:33:15.230782 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:15.230740 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ct9t6" Jul 09 04:33:15.240958 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:15.240928 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:15.344270 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:33:15.344189 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0a12812_bbc1_4b88_a2f2_858d22d67e64.slice/crio-b4b4bddfc771d614f9794c89469e3ecc8f23ec3747bf38a6d82cd5e471fe7d63 WatchSource:0}: Error finding container b4b4bddfc771d614f9794c89469e3ecc8f23ec3747bf38a6d82cd5e471fe7d63: Status 404 returned error can't find the container with id b4b4bddfc771d614f9794c89469e3ecc8f23ec3747bf38a6d82cd5e471fe7d63 Jul 09 04:33:15.344983 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:33:15.344934 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc88f9f0_1d2f_4522_b7ca_d0b9104f85fa.slice/crio-f84b15849bfa4cfc80c48b7e7c7b575249f3b0514097640b6dea06a105f70e1a WatchSource:0}: Error finding container f84b15849bfa4cfc80c48b7e7c7b575249f3b0514097640b6dea06a105f70e1a: Status 404 returned error can't find the container with id f84b15849bfa4cfc80c48b7e7c7b575249f3b0514097640b6dea06a105f70e1a Jul 09 04:33:15.462834 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:15.462808 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ct9t6"] Jul 09 04:33:15.464620 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:33:15.464593 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a6fbfe4_8f20_4ae6_98bd_9a197b6b5766.slice/crio-5b7bd5d2a66d17db2bfc9f03f083ba0cc7bfeaaf17ed2ab0c946fdef6f388a15 WatchSource:0}: Error finding container 5b7bd5d2a66d17db2bfc9f03f083ba0cc7bfeaaf17ed2ab0c946fdef6f388a15: Status 404 returned error can't find the container with id 5b7bd5d2a66d17db2bfc9f03f083ba0cc7bfeaaf17ed2ab0c946fdef6f388a15 Jul 09 04:33:15.581620 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:15.581538 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bgsl7" event={"ID":"f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59","Type":"ContainerStarted","Data":"030eb68207bd0c599606fec7f046aecb787072a40fc4d904a9abf2d0cf559053"} Jul 09 04:33:15.582626 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:15.582598 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ct9t6" event={"ID":"7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766","Type":"ContainerStarted","Data":"5b7bd5d2a66d17db2bfc9f03f083ba0cc7bfeaaf17ed2ab0c946fdef6f388a15"} Jul 09 04:33:15.584070 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:15.584041 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zl4tp" event={"ID":"d0a12812-bbc1-4b88-a2f2-858d22d67e64","Type":"ContainerStarted","Data":"b4b4bddfc771d614f9794c89469e3ecc8f23ec3747bf38a6d82cd5e471fe7d63"} Jul 09 04:33:15.585065 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:15.585043 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gpx47" event={"ID":"dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa","Type":"ContainerStarted","Data":"f84b15849bfa4cfc80c48b7e7c7b575249f3b0514097640b6dea06a105f70e1a"} Jul 09 04:33:16.514967 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:16.514904 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:16.591025 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:16.590979 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-596786469b-l988v" event={"ID":"0bc1140e-97ff-4c83-a076-127486b1b83c","Type":"ContainerStarted","Data":"cccd697bab75d24a79d40f9455d5356088e049056a4c47075a2454674b994464"} Jul 09 04:33:16.591186 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:16.591044 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-596786469b-l988v" event={"ID":"0bc1140e-97ff-4c83-a076-127486b1b83c","Type":"ContainerStarted","Data":"20133f510a737739ac2847d0ca0715ac1a2731869e25710e9430f4eba939503a"} Jul 09 04:33:16.591186 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:16.591061 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-596786469b-l988v" event={"ID":"0bc1140e-97ff-4c83-a076-127486b1b83c","Type":"ContainerStarted","Data":"2205563074a8f23cbcb1d09a15bd605d523ac09ea084c1442f7b993343de13ae"} Jul 09 04:33:16.610549 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:16.610304 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-596786469b-l988v" podStartSLOduration=1.570042018 podStartE2EDuration="5.610279891s" totalImagesPullingTime="4.040237873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:33:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:33:16.609189727 +0000 UTC m=+68.860713306" watchObservedRunningTime="2026-07-09 04:33:16.610279891 +0000 UTC m=+68.861803470" Jul 09 04:33:17.597536 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:17.597489 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gpx47" event={"ID":"dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa","Type":"ContainerStarted","Data":"c44f5c0f34a2bc0b9c19b2362057c953744b36184230fa12344c506366cd459a"} Jul 09 04:33:18.460584 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:18.460545 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/359f3d6e-a08a-487d-a480-a642dad64ec4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xgznh\" (UID: \"359f3d6e-a08a-487d-a480-a642dad64ec4\") " pod="openshift-insights/insights-runtime-extractor-xgznh" Jul 09 04:33:18.462981 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:18.462953 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/359f3d6e-a08a-487d-a480-a642dad64ec4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xgznh\" (UID: \"359f3d6e-a08a-487d-a480-a642dad64ec4\") " pod="openshift-insights/insights-runtime-extractor-xgznh" Jul 09 04:33:18.602638 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:18.602600 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ct9t6" event={"ID":"7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766","Type":"ContainerStarted","Data":"2fee01010f6404e255f2d54e86870021105694ede8a112fc7c3a0f4533649b03"} Jul 09 04:33:18.604498 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:18.604467 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gpx47" event={"ID":"dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa","Type":"ContainerStarted","Data":"9c338ff55ea57adf6c6b413d664511512683f4ff1772868c0212c2c1b2f6cc09"} Jul 09 04:33:18.619743 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:18.619694 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-gpx47" podStartSLOduration=68.930005333 podStartE2EDuration="1m10.619676048s" totalImagesPullingTime="1.689670715s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:32:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:33:18.619449671 +0000 UTC m=+70.870973250" watchObservedRunningTime="2026-07-09 04:33:18.619676048 +0000 UTC m=+70.871199627" Jul 09 04:33:18.689518 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:18.689480 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:18.737083 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:18.736981 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-c4pmv\"" Jul 09 04:33:18.745150 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:18.745108 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xgznh" Jul 09 04:33:18.790375 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:18.790328 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:18.847840 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:18.847805 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:19.982199 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:19.982162 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:20.049921 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:20.049877 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:20.213997 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:20.213959 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:20.496272 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:20.496246 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xgznh"] Jul 09 04:33:20.499615 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:33:20.499590 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod359f3d6e_a08a_487d_a480_a642dad64ec4.slice/crio-02db74d9ac93f77f241c57de774dc62e822ee9a09855377d720ccd1e48bd4851 WatchSource:0}: Error finding container 02db74d9ac93f77f241c57de774dc62e822ee9a09855377d720ccd1e48bd4851: Status 404 returned error can't find the container with id 02db74d9ac93f77f241c57de774dc62e822ee9a09855377d720ccd1e48bd4851 Jul 09 04:33:20.614256 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:20.614151 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bgsl7" event={"ID":"f7c2ccf6-9a6e-447f-9e20-ecb28fbdac59","Type":"ContainerStarted","Data":"5d7a2ebcdde7a93fa0291c23884e325a9fef52eff29ddbf1aa52a0b41348ce10"} Jul 09 04:33:20.614256 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:20.614239 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-bgsl7" Jul 09 04:33:20.616040 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:20.616010 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ct9t6" event={"ID":"7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766","Type":"ContainerStarted","Data":"5c6ac21f7239a059aa01f9262ac5108da9b3610b87e0c40ad8210ef45c516ad6"} Jul 09 04:33:20.616153 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:20.616076 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-ct9t6" Jul 09 04:33:20.617400 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:20.617374 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zl4tp" event={"ID":"d0a12812-bbc1-4b88-a2f2-858d22d67e64","Type":"ContainerStarted","Data":"e426e69ea36a62a1319f494c15ba121f06b732755b0c54002522978d9a5b0894"} Jul 09 04:33:20.618481 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:20.618463 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xgznh" event={"ID":"359f3d6e-a08a-487d-a480-a642dad64ec4","Type":"ContainerStarted","Data":"02db74d9ac93f77f241c57de774dc62e822ee9a09855377d720ccd1e48bd4851"} Jul 09 04:33:20.629130 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:20.629080 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-bgsl7" podStartSLOduration=67.619956995 podStartE2EDuration="1m12.629067809s" totalImagesPullingTime="5.009110814s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:32:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:33:20.628071273 +0000 UTC m=+72.879594852" watchObservedRunningTime="2026-07-09 04:33:20.629067809 +0000 UTC m=+72.880591378" Jul 09 04:33:20.641968 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:20.641921 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-zl4tp" podStartSLOduration=67.621232045 podStartE2EDuration="1m12.641905572s" totalImagesPullingTime="5.020673527s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:32:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:33:20.641252805 +0000 UTC m=+72.892776384" watchObservedRunningTime="2026-07-09 04:33:20.641905572 +0000 UTC m=+72.893429150" Jul 09 04:33:20.656578 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:20.656527 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-ct9t6" podStartSLOduration=35.310424148 podStartE2EDuration="37.656507789s" totalImagesPullingTime="2.346083641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:32:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:33:20.655753042 +0000 UTC m=+72.907276621" watchObservedRunningTime="2026-07-09 04:33:20.656507789 +0000 UTC m=+72.908031366" Jul 09 04:33:22.970398 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:22.970356 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:23.020177 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:23.020144 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:24.635144 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:24.635108 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xgznh" event={"ID":"359f3d6e-a08a-487d-a480-a642dad64ec4","Type":"ContainerStarted","Data":"81ff3ee4a25552c7c3a0dddbb9d00ef2acb7bb048860e9791d37908fbcbd4b7a"} Jul 09 04:33:25.145142 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:25.145109 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:25.248089 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:25.248011 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:25.483976 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:25.483940 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:25.639972 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:25.639886 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xgznh" event={"ID":"359f3d6e-a08a-487d-a480-a642dad64ec4","Type":"ContainerStarted","Data":"3dabbbbe54274161fda05415991396f3f7ae9832ac6d26b0db7712eeb8528be3"} Jul 09 04:33:27.625176 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:27.625140 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-ct9t6" Jul 09 04:33:27.885229 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:27.885153 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:28.118664 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.118634 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:28.341019 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.340984 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:28.355315 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.355288 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-swnzm"] Jul 09 04:33:28.358304 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.358289 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-swnzm" Jul 09 04:33:28.360866 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.360844 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 09 04:33:28.360982 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.360886 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 09 04:33:28.360982 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.360907 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 09 04:33:28.361925 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.361906 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 09 04:33:28.362023 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.361962 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-zl8qw\"" Jul 09 04:33:28.362023 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.361967 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 09 04:33:28.373561 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.372862 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-swnzm"] Jul 09 04:33:28.373561 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.372974 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6slcp"] Jul 09 04:33:28.376976 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.376957 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6slcp" Jul 09 04:33:28.378606 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.378586 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-mvq75"] Jul 09 04:33:28.379598 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.379579 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-cxmjg\"" Jul 09 04:33:28.379598 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.379592 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 09 04:33:28.379738 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.379604 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 09 04:33:28.379738 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.379583 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 09 04:33:28.381445 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.381428 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-mvq75" Jul 09 04:33:28.383865 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.383847 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 09 04:33:28.384097 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.384064 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-nf8lp\"" Jul 09 04:33:28.384240 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.384074 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 09 04:33:28.385283 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.385265 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 09 04:33:28.393765 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.393742 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-mvq75"] Jul 09 04:33:28.425872 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.425841 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:28.438919 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.438895 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3867ddde-c7e8-4e28-b44f-ce053ff6e7f9-sys\") pod \"node-exporter-6slcp\" (UID: \"3867ddde-c7e8-4e28-b44f-ce053ff6e7f9\") " pod="openshift-monitoring/node-exporter-6slcp" Jul 09 04:33:28.439018 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.438926 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3867ddde-c7e8-4e28-b44f-ce053ff6e7f9-node-exporter-tls\") pod \"node-exporter-6slcp\" (UID: \"3867ddde-c7e8-4e28-b44f-ce053ff6e7f9\") " pod="openshift-monitoring/node-exporter-6slcp" Jul 09 04:33:28.439018 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.438948 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ffccf1f2-c6d1-4d17-8387-a38651058f81-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-mvq75\" (UID: \"ffccf1f2-c6d1-4d17-8387-a38651058f81\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-mvq75" Jul 09 04:33:28.439018 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.438995 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/10318b5f-09fb-4670-9c39-a6f028c961fb-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-swnzm\" (UID: \"10318b5f-09fb-4670-9c39-a6f028c961fb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-swnzm" Jul 09 04:33:28.439147 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.439022 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/10318b5f-09fb-4670-9c39-a6f028c961fb-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-swnzm\" (UID: \"10318b5f-09fb-4670-9c39-a6f028c961fb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-swnzm" Jul 09 04:33:28.439147 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.439056 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3867ddde-c7e8-4e28-b44f-ce053ff6e7f9-node-exporter-wtmp\") pod \"node-exporter-6slcp\" (UID: \"3867ddde-c7e8-4e28-b44f-ce053ff6e7f9\") " pod="openshift-monitoring/node-exporter-6slcp" Jul 09 04:33:28.439147 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.439074 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3867ddde-c7e8-4e28-b44f-ce053ff6e7f9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6slcp\" (UID: \"3867ddde-c7e8-4e28-b44f-ce053ff6e7f9\") " pod="openshift-monitoring/node-exporter-6slcp" Jul 09 04:33:28.439147 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.439094 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ffccf1f2-c6d1-4d17-8387-a38651058f81-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-mvq75\" (UID: \"ffccf1f2-c6d1-4d17-8387-a38651058f81\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-mvq75" Jul 09 04:33:28.439147 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.439110 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3867ddde-c7e8-4e28-b44f-ce053ff6e7f9-node-exporter-textfile\") pod \"node-exporter-6slcp\" (UID: \"3867ddde-c7e8-4e28-b44f-ce053ff6e7f9\") " pod="openshift-monitoring/node-exporter-6slcp" Jul 09 04:33:28.439147 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.439125 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3867ddde-c7e8-4e28-b44f-ce053ff6e7f9-node-exporter-accelerators-collector-config\") pod \"node-exporter-6slcp\" (UID: \"3867ddde-c7e8-4e28-b44f-ce053ff6e7f9\") " pod="openshift-monitoring/node-exporter-6slcp" Jul 09 04:33:28.439147 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.439142 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gchs\" (UniqueName: \"kubernetes.io/projected/3867ddde-c7e8-4e28-b44f-ce053ff6e7f9-kube-api-access-8gchs\") pod \"node-exporter-6slcp\" (UID: \"3867ddde-c7e8-4e28-b44f-ce053ff6e7f9\") " pod="openshift-monitoring/node-exporter-6slcp" Jul 09 04:33:28.439387 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.439159 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ffccf1f2-c6d1-4d17-8387-a38651058f81-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-mvq75\" (UID: \"ffccf1f2-c6d1-4d17-8387-a38651058f81\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-mvq75" Jul 09 04:33:28.439387 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.439173 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ffccf1f2-c6d1-4d17-8387-a38651058f81-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-mvq75\" (UID: \"ffccf1f2-c6d1-4d17-8387-a38651058f81\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-mvq75" Jul 09 04:33:28.439387 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.439197 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7qdh\" (UniqueName: \"kubernetes.io/projected/ffccf1f2-c6d1-4d17-8387-a38651058f81-kube-api-access-k7qdh\") pod \"kube-state-metrics-7c9dbb8b7d-mvq75\" (UID: \"ffccf1f2-c6d1-4d17-8387-a38651058f81\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-mvq75" Jul 09 04:33:28.439387 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.439277 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/10318b5f-09fb-4670-9c39-a6f028c961fb-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-swnzm\" (UID: \"10318b5f-09fb-4670-9c39-a6f028c961fb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-swnzm" Jul 09 04:33:28.439387 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.439303 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3867ddde-c7e8-4e28-b44f-ce053ff6e7f9-metrics-client-ca\") pod \"node-exporter-6slcp\" (UID: \"3867ddde-c7e8-4e28-b44f-ce053ff6e7f9\") " pod="openshift-monitoring/node-exporter-6slcp" Jul 09 04:33:28.439387 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.439332 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ffccf1f2-c6d1-4d17-8387-a38651058f81-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-mvq75\" (UID: \"ffccf1f2-c6d1-4d17-8387-a38651058f81\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-mvq75" Jul 09 04:33:28.439387 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.439369 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3867ddde-c7e8-4e28-b44f-ce053ff6e7f9-root\") pod \"node-exporter-6slcp\" (UID: \"3867ddde-c7e8-4e28-b44f-ce053ff6e7f9\") " pod="openshift-monitoring/node-exporter-6slcp" Jul 09 04:33:28.439638 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.439398 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4q4c\" (UniqueName: \"kubernetes.io/projected/10318b5f-09fb-4670-9c39-a6f028c961fb-kube-api-access-w4q4c\") pod \"openshift-state-metrics-54cf96dfcf-swnzm\" (UID: \"10318b5f-09fb-4670-9c39-a6f028c961fb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-swnzm" Jul 09 04:33:28.540490 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.540450 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3867ddde-c7e8-4e28-b44f-ce053ff6e7f9-node-exporter-wtmp\") pod \"node-exporter-6slcp\" (UID: \"3867ddde-c7e8-4e28-b44f-ce053ff6e7f9\") " pod="openshift-monitoring/node-exporter-6slcp" Jul 09 04:33:28.540490 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.540489 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3867ddde-c7e8-4e28-b44f-ce053ff6e7f9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6slcp\" (UID: \"3867ddde-c7e8-4e28-b44f-ce053ff6e7f9\") " pod="openshift-monitoring/node-exporter-6slcp" Jul 09 04:33:28.540739 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.540508 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ffccf1f2-c6d1-4d17-8387-a38651058f81-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-mvq75\" (UID: \"ffccf1f2-c6d1-4d17-8387-a38651058f81\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-mvq75" Jul 09 04:33:28.540739 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.540526 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3867ddde-c7e8-4e28-b44f-ce053ff6e7f9-node-exporter-textfile\") pod \"node-exporter-6slcp\" (UID: \"3867ddde-c7e8-4e28-b44f-ce053ff6e7f9\") " pod="openshift-monitoring/node-exporter-6slcp" Jul 09 04:33:28.540739 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.540550 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3867ddde-c7e8-4e28-b44f-ce053ff6e7f9-node-exporter-accelerators-collector-config\") pod \"node-exporter-6slcp\" (UID: \"3867ddde-c7e8-4e28-b44f-ce053ff6e7f9\") " pod="openshift-monitoring/node-exporter-6slcp" Jul 09 04:33:28.540739 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.540573 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gchs\" (UniqueName: \"kubernetes.io/projected/3867ddde-c7e8-4e28-b44f-ce053ff6e7f9-kube-api-access-8gchs\") pod \"node-exporter-6slcp\" (UID: \"3867ddde-c7e8-4e28-b44f-ce053ff6e7f9\") " pod="openshift-monitoring/node-exporter-6slcp" Jul 09 04:33:28.540739 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.540633 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3867ddde-c7e8-4e28-b44f-ce053ff6e7f9-node-exporter-wtmp\") pod \"node-exporter-6slcp\" (UID: \"3867ddde-c7e8-4e28-b44f-ce053ff6e7f9\") " pod="openshift-monitoring/node-exporter-6slcp" Jul 09 04:33:28.540739 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.540663 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ffccf1f2-c6d1-4d17-8387-a38651058f81-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-mvq75\" (UID: \"ffccf1f2-c6d1-4d17-8387-a38651058f81\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-mvq75" Jul 09 04:33:28.540739 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.540695 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ffccf1f2-c6d1-4d17-8387-a38651058f81-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-mvq75\" (UID: \"ffccf1f2-c6d1-4d17-8387-a38651058f81\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-mvq75" Jul 09 04:33:28.541071 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.540749 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7qdh\" (UniqueName: \"kubernetes.io/projected/ffccf1f2-c6d1-4d17-8387-a38651058f81-kube-api-access-k7qdh\") pod \"kube-state-metrics-7c9dbb8b7d-mvq75\" (UID: \"ffccf1f2-c6d1-4d17-8387-a38651058f81\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-mvq75" Jul 09 04:33:28.541071 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.540796 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/10318b5f-09fb-4670-9c39-a6f028c961fb-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-swnzm\" (UID: \"10318b5f-09fb-4670-9c39-a6f028c961fb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-swnzm" Jul 09 04:33:28.541071 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.540827 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3867ddde-c7e8-4e28-b44f-ce053ff6e7f9-metrics-client-ca\") pod \"node-exporter-6slcp\" (UID: \"3867ddde-c7e8-4e28-b44f-ce053ff6e7f9\") " pod="openshift-monitoring/node-exporter-6slcp" Jul 09 04:33:28.541071 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.540869 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ffccf1f2-c6d1-4d17-8387-a38651058f81-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-mvq75\" (UID: \"ffccf1f2-c6d1-4d17-8387-a38651058f81\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-mvq75" Jul 09 04:33:28.541071 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.540897 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3867ddde-c7e8-4e28-b44f-ce053ff6e7f9-root\") pod \"node-exporter-6slcp\" (UID: \"3867ddde-c7e8-4e28-b44f-ce053ff6e7f9\") " pod="openshift-monitoring/node-exporter-6slcp" Jul 09 04:33:28.541071 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.540925 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3867ddde-c7e8-4e28-b44f-ce053ff6e7f9-node-exporter-textfile\") pod \"node-exporter-6slcp\" (UID: \"3867ddde-c7e8-4e28-b44f-ce053ff6e7f9\") " pod="openshift-monitoring/node-exporter-6slcp" Jul 09 04:33:28.541071 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.540929 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4q4c\" (UniqueName: \"kubernetes.io/projected/10318b5f-09fb-4670-9c39-a6f028c961fb-kube-api-access-w4q4c\") pod \"openshift-state-metrics-54cf96dfcf-swnzm\" (UID: \"10318b5f-09fb-4670-9c39-a6f028c961fb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-swnzm" Jul 09 04:33:28.541071 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.540998 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3867ddde-c7e8-4e28-b44f-ce053ff6e7f9-sys\") pod \"node-exporter-6slcp\" (UID: \"3867ddde-c7e8-4e28-b44f-ce053ff6e7f9\") " pod="openshift-monitoring/node-exporter-6slcp" Jul 09 04:33:28.541071 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:33:28.541013 2565 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 09 04:33:28.541071 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.541031 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3867ddde-c7e8-4e28-b44f-ce053ff6e7f9-node-exporter-tls\") pod \"node-exporter-6slcp\" (UID: \"3867ddde-c7e8-4e28-b44f-ce053ff6e7f9\") " pod="openshift-monitoring/node-exporter-6slcp" Jul 09 04:33:28.541071 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.541064 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ffccf1f2-c6d1-4d17-8387-a38651058f81-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-mvq75\" (UID: \"ffccf1f2-c6d1-4d17-8387-a38651058f81\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-mvq75" Jul 09 04:33:28.541071 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:33:28.541072 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffccf1f2-c6d1-4d17-8387-a38651058f81-kube-state-metrics-tls podName:ffccf1f2-c6d1-4d17-8387-a38651058f81 nodeName:}" failed. No retries permitted until 2026-07-09 04:33:29.041052112 +0000 UTC m=+81.292575670 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/ffccf1f2-c6d1-4d17-8387-a38651058f81-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-mvq75" (UID: "ffccf1f2-c6d1-4d17-8387-a38651058f81") : secret "kube-state-metrics-tls" not found Jul 09 04:33:28.541671 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.541092 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ffccf1f2-c6d1-4d17-8387-a38651058f81-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-mvq75\" (UID: \"ffccf1f2-c6d1-4d17-8387-a38651058f81\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-mvq75" Jul 09 04:33:28.541671 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.541117 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3867ddde-c7e8-4e28-b44f-ce053ff6e7f9-sys\") pod \"node-exporter-6slcp\" (UID: \"3867ddde-c7e8-4e28-b44f-ce053ff6e7f9\") " pod="openshift-monitoring/node-exporter-6slcp" Jul 09 04:33:28.541671 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:33:28.541121 2565 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 09 04:33:28.541671 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.541128 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/10318b5f-09fb-4670-9c39-a6f028c961fb-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-swnzm\" (UID: \"10318b5f-09fb-4670-9c39-a6f028c961fb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-swnzm" Jul 09 04:33:28.541671 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.541156 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/10318b5f-09fb-4670-9c39-a6f028c961fb-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-swnzm\" (UID: \"10318b5f-09fb-4670-9c39-a6f028c961fb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-swnzm" Jul 09 04:33:28.541671 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:33:28.541198 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3867ddde-c7e8-4e28-b44f-ce053ff6e7f9-node-exporter-tls podName:3867ddde-c7e8-4e28-b44f-ce053ff6e7f9 nodeName:}" failed. No retries permitted until 2026-07-09 04:33:29.041186623 +0000 UTC m=+81.292710196 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/3867ddde-c7e8-4e28-b44f-ce053ff6e7f9-node-exporter-tls") pod "node-exporter-6slcp" (UID: "3867ddde-c7e8-4e28-b44f-ce053ff6e7f9") : secret "node-exporter-tls" not found Jul 09 04:33:28.541671 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.541236 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3867ddde-c7e8-4e28-b44f-ce053ff6e7f9-root\") pod \"node-exporter-6slcp\" (UID: \"3867ddde-c7e8-4e28-b44f-ce053ff6e7f9\") " pod="openshift-monitoring/node-exporter-6slcp" Jul 09 04:33:28.541671 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.541424 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3867ddde-c7e8-4e28-b44f-ce053ff6e7f9-metrics-client-ca\") pod \"node-exporter-6slcp\" (UID: \"3867ddde-c7e8-4e28-b44f-ce053ff6e7f9\") " pod="openshift-monitoring/node-exporter-6slcp" Jul 09 04:33:28.542059 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.541866 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ffccf1f2-c6d1-4d17-8387-a38651058f81-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-mvq75\" (UID: \"ffccf1f2-c6d1-4d17-8387-a38651058f81\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-mvq75" Jul 09 04:33:28.542170 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.542091 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/10318b5f-09fb-4670-9c39-a6f028c961fb-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-swnzm\" (UID: \"10318b5f-09fb-4670-9c39-a6f028c961fb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-swnzm" Jul 09 04:33:28.542397 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.542373 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ffccf1f2-c6d1-4d17-8387-a38651058f81-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-mvq75\" (UID: \"ffccf1f2-c6d1-4d17-8387-a38651058f81\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-mvq75" Jul 09 04:33:28.542480 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.542381 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3867ddde-c7e8-4e28-b44f-ce053ff6e7f9-node-exporter-accelerators-collector-config\") pod \"node-exporter-6slcp\" (UID: \"3867ddde-c7e8-4e28-b44f-ce053ff6e7f9\") " pod="openshift-monitoring/node-exporter-6slcp" Jul 09 04:33:28.543299 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.543281 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3867ddde-c7e8-4e28-b44f-ce053ff6e7f9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6slcp\" (UID: \"3867ddde-c7e8-4e28-b44f-ce053ff6e7f9\") " pod="openshift-monitoring/node-exporter-6slcp" Jul 09 04:33:28.543485 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.543464 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ffccf1f2-c6d1-4d17-8387-a38651058f81-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-mvq75\" (UID: \"ffccf1f2-c6d1-4d17-8387-a38651058f81\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-mvq75" Jul 09 04:33:28.544067 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.544043 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/10318b5f-09fb-4670-9c39-a6f028c961fb-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-swnzm\" (UID: \"10318b5f-09fb-4670-9c39-a6f028c961fb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-swnzm" Jul 09 04:33:28.544146 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.544106 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/10318b5f-09fb-4670-9c39-a6f028c961fb-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-swnzm\" (UID: \"10318b5f-09fb-4670-9c39-a6f028c961fb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-swnzm" Jul 09 04:33:28.550188 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.550146 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gchs\" (UniqueName: \"kubernetes.io/projected/3867ddde-c7e8-4e28-b44f-ce053ff6e7f9-kube-api-access-8gchs\") pod \"node-exporter-6slcp\" (UID: \"3867ddde-c7e8-4e28-b44f-ce053ff6e7f9\") " pod="openshift-monitoring/node-exporter-6slcp" Jul 09 04:33:28.550188 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.550179 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4q4c\" (UniqueName: \"kubernetes.io/projected/10318b5f-09fb-4670-9c39-a6f028c961fb-kube-api-access-w4q4c\") pod \"openshift-state-metrics-54cf96dfcf-swnzm\" (UID: \"10318b5f-09fb-4670-9c39-a6f028c961fb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-swnzm" Jul 09 04:33:28.550867 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.550847 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7qdh\" (UniqueName: \"kubernetes.io/projected/ffccf1f2-c6d1-4d17-8387-a38651058f81-kube-api-access-k7qdh\") pod \"kube-state-metrics-7c9dbb8b7d-mvq75\" (UID: \"ffccf1f2-c6d1-4d17-8387-a38651058f81\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-mvq75" Jul 09 04:33:28.551695 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.551674 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:28.651082 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.650972 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:28.652435 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.652409 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xgznh" event={"ID":"359f3d6e-a08a-487d-a480-a642dad64ec4","Type":"ContainerStarted","Data":"4beb3ceffa09fda829e196dfbcaaf31655c54f09373d55bafe9cdd532c2eb594"} Jul 09 04:33:28.666825 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.666801 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-swnzm" Jul 09 04:33:28.669496 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.669447 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-xgznh" podStartSLOduration=35.424422204 podStartE2EDuration="42.669430646s" totalImagesPullingTime="7.245008442s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:32:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:33:28.669015944 +0000 UTC m=+80.920539522" watchObservedRunningTime="2026-07-09 04:33:28.669430646 +0000 UTC m=+80.920954225" Jul 09 04:33:28.751891 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.751857 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:28.796708 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.796558 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-swnzm"] Jul 09 04:33:28.799188 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:33:28.799164 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10318b5f_09fb_4670_9c39_a6f028c961fb.slice/crio-42762536dbfed89ab0275f93cae805fcdb52bd0fdc098e1b839af7ec5979f0d5 WatchSource:0}: Error finding container 42762536dbfed89ab0275f93cae805fcdb52bd0fdc098e1b839af7ec5979f0d5: Status 404 returned error can't find the container with id 42762536dbfed89ab0275f93cae805fcdb52bd0fdc098e1b839af7ec5979f0d5 Jul 09 04:33:28.896782 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.896758 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:28.996703 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:28.996653 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:29.045988 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:29.045951 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3867ddde-c7e8-4e28-b44f-ce053ff6e7f9-node-exporter-tls\") pod \"node-exporter-6slcp\" (UID: \"3867ddde-c7e8-4e28-b44f-ce053ff6e7f9\") " pod="openshift-monitoring/node-exporter-6slcp" Jul 09 04:33:29.046124 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:29.046045 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ffccf1f2-c6d1-4d17-8387-a38651058f81-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-mvq75\" (UID: \"ffccf1f2-c6d1-4d17-8387-a38651058f81\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-mvq75" Jul 09 04:33:29.048320 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:29.048291 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3867ddde-c7e8-4e28-b44f-ce053ff6e7f9-node-exporter-tls\") pod \"node-exporter-6slcp\" (UID: \"3867ddde-c7e8-4e28-b44f-ce053ff6e7f9\") " pod="openshift-monitoring/node-exporter-6slcp" Jul 09 04:33:29.048463 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:29.048399 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ffccf1f2-c6d1-4d17-8387-a38651058f81-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-mvq75\" (UID: \"ffccf1f2-c6d1-4d17-8387-a38651058f81\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-mvq75" Jul 09 04:33:29.287649 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:29.287564 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6slcp" Jul 09 04:33:29.292626 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:29.292604 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-mvq75" Jul 09 04:33:29.299412 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:33:29.299388 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3867ddde_c7e8_4e28_b44f_ce053ff6e7f9.slice/crio-29c74b809370ee4516d8d57b00804fb44555da92574b2c7c93e012df7bfaf4ae WatchSource:0}: Error finding container 29c74b809370ee4516d8d57b00804fb44555da92574b2c7c93e012df7bfaf4ae: Status 404 returned error can't find the container with id 29c74b809370ee4516d8d57b00804fb44555da92574b2c7c93e012df7bfaf4ae Jul 09 04:33:29.344880 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:29.344844 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:29.427631 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:29.427600 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-mvq75"] Jul 09 04:33:29.428604 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:29.428581 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:29.430763 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:33:29.430740 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffccf1f2_c6d1_4d17_8387_a38651058f81.slice/crio-b13b4b56d362042becd9ba4e0ed3290b94572a07eeb1c92994567315f2811264 WatchSource:0}: Error finding container b13b4b56d362042becd9ba4e0ed3290b94572a07eeb1c92994567315f2811264: Status 404 returned error can't find the container with id b13b4b56d362042becd9ba4e0ed3290b94572a07eeb1c92994567315f2811264 Jul 09 04:33:29.487622 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:29.487588 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:29.658421 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:29.658383 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-swnzm" event={"ID":"10318b5f-09fb-4670-9c39-a6f028c961fb","Type":"ContainerStarted","Data":"70e4564c9de0df8b9be7567277785643bb8711cefb5ab40c5e74d8903e9895f0"} Jul 09 04:33:29.658866 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:29.658467 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-swnzm" event={"ID":"10318b5f-09fb-4670-9c39-a6f028c961fb","Type":"ContainerStarted","Data":"e5c3eccf013c649b3d91055255361a4732408962db450f153c478b863d4f4ee9"} Jul 09 04:33:29.658866 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:29.658498 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-swnzm" event={"ID":"10318b5f-09fb-4670-9c39-a6f028c961fb","Type":"ContainerStarted","Data":"42762536dbfed89ab0275f93cae805fcdb52bd0fdc098e1b839af7ec5979f0d5"} Jul 09 04:33:29.659760 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:29.659729 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-mvq75" event={"ID":"ffccf1f2-c6d1-4d17-8387-a38651058f81","Type":"ContainerStarted","Data":"b13b4b56d362042becd9ba4e0ed3290b94572a07eeb1c92994567315f2811264"} Jul 09 04:33:29.661135 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:29.661113 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6slcp" event={"ID":"3867ddde-c7e8-4e28-b44f-ce053ff6e7f9","Type":"ContainerStarted","Data":"29c74b809370ee4516d8d57b00804fb44555da92574b2c7c93e012df7bfaf4ae"} Jul 09 04:33:29.746315 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:29.746192 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:29.896671 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:29.896638 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:30.668693 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:30.668657 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-swnzm" event={"ID":"10318b5f-09fb-4670-9c39-a6f028c961fb","Type":"ContainerStarted","Data":"ab2c9fd44d831b488fce9c20bd66151f661d11d09ef2cb7087bea916c7912239"} Jul 09 04:33:30.686181 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:30.686148 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-67547c88b6-pxvpl"] Jul 09 04:33:30.689348 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:30.689310 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67547c88b6-pxvpl" Jul 09 04:33:30.691132 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:30.691086 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-swnzm" podStartSLOduration=1.282301568 podStartE2EDuration="2.691069829s" totalImagesPullingTime="1.408768261s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:33:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:33:30.690309252 +0000 UTC m=+82.941832827" watchObservedRunningTime="2026-07-09 04:33:30.691069829 +0000 UTC m=+82.942593406" Jul 09 04:33:30.692186 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:30.692168 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 09 04:33:30.692301 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:30.692207 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 09 04:33:30.692612 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:30.692564 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 09 04:33:30.692612 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:30.692596 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 09 04:33:30.692612 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:30.692606 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 09 04:33:30.692823 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:30.692623 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 09 04:33:30.692935 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:30.692918 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 09 04:33:30.693006 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:30.692953 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-k7dhp\"" Jul 09 04:33:30.697486 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:30.697413 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 09 04:33:30.700937 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:30.700917 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67547c88b6-pxvpl"] Jul 09 04:33:30.760847 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:30.760813 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mtjs\" (UniqueName: \"kubernetes.io/projected/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-kube-api-access-9mtjs\") pod \"console-67547c88b6-pxvpl\" (UID: \"d2b8e7bd-d841-494f-b1c1-d182f7957ebb\") " pod="openshift-console/console-67547c88b6-pxvpl" Jul 09 04:33:30.761036 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:30.760931 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-console-config\") pod \"console-67547c88b6-pxvpl\" (UID: \"d2b8e7bd-d841-494f-b1c1-d182f7957ebb\") " pod="openshift-console/console-67547c88b6-pxvpl" Jul 09 04:33:30.761036 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:30.760972 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-trusted-ca-bundle\") pod \"console-67547c88b6-pxvpl\" (UID: \"d2b8e7bd-d841-494f-b1c1-d182f7957ebb\") " pod="openshift-console/console-67547c88b6-pxvpl" Jul 09 04:33:30.761159 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:30.761068 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-oauth-serving-cert\") pod \"console-67547c88b6-pxvpl\" (UID: \"d2b8e7bd-d841-494f-b1c1-d182f7957ebb\") " pod="openshift-console/console-67547c88b6-pxvpl" Jul 09 04:33:30.761159 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:30.761114 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-console-serving-cert\") pod \"console-67547c88b6-pxvpl\" (UID: \"d2b8e7bd-d841-494f-b1c1-d182f7957ebb\") " pod="openshift-console/console-67547c88b6-pxvpl" Jul 09 04:33:30.761255 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:30.761230 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-console-oauth-config\") pod \"console-67547c88b6-pxvpl\" (UID: \"d2b8e7bd-d841-494f-b1c1-d182f7957ebb\") " pod="openshift-console/console-67547c88b6-pxvpl" Jul 09 04:33:30.761312 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:30.761267 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-service-ca\") pod \"console-67547c88b6-pxvpl\" (UID: \"d2b8e7bd-d841-494f-b1c1-d182f7957ebb\") " pod="openshift-console/console-67547c88b6-pxvpl" Jul 09 04:33:30.862259 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:30.862225 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-console-oauth-config\") pod \"console-67547c88b6-pxvpl\" (UID: \"d2b8e7bd-d841-494f-b1c1-d182f7957ebb\") " pod="openshift-console/console-67547c88b6-pxvpl" Jul 09 04:33:30.862259 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:30.862263 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-service-ca\") pod \"console-67547c88b6-pxvpl\" (UID: \"d2b8e7bd-d841-494f-b1c1-d182f7957ebb\") " pod="openshift-console/console-67547c88b6-pxvpl" Jul 09 04:33:30.862532 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:30.862290 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mtjs\" (UniqueName: \"kubernetes.io/projected/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-kube-api-access-9mtjs\") pod \"console-67547c88b6-pxvpl\" (UID: \"d2b8e7bd-d841-494f-b1c1-d182f7957ebb\") " pod="openshift-console/console-67547c88b6-pxvpl" Jul 09 04:33:30.862532 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:30.862313 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-console-config\") pod \"console-67547c88b6-pxvpl\" (UID: \"d2b8e7bd-d841-494f-b1c1-d182f7957ebb\") " pod="openshift-console/console-67547c88b6-pxvpl" Jul 09 04:33:30.862532 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:30.862359 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-trusted-ca-bundle\") pod \"console-67547c88b6-pxvpl\" (UID: \"d2b8e7bd-d841-494f-b1c1-d182f7957ebb\") " pod="openshift-console/console-67547c88b6-pxvpl" Jul 09 04:33:30.862532 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:30.862426 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-oauth-serving-cert\") pod \"console-67547c88b6-pxvpl\" (UID: \"d2b8e7bd-d841-494f-b1c1-d182f7957ebb\") " pod="openshift-console/console-67547c88b6-pxvpl" Jul 09 04:33:30.862532 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:30.862452 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-console-serving-cert\") pod \"console-67547c88b6-pxvpl\" (UID: \"d2b8e7bd-d841-494f-b1c1-d182f7957ebb\") " pod="openshift-console/console-67547c88b6-pxvpl" Jul 09 04:33:30.863140 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:30.863111 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-console-config\") pod \"console-67547c88b6-pxvpl\" (UID: \"d2b8e7bd-d841-494f-b1c1-d182f7957ebb\") " pod="openshift-console/console-67547c88b6-pxvpl" Jul 09 04:33:30.863246 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:30.863152 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-service-ca\") pod \"console-67547c88b6-pxvpl\" (UID: \"d2b8e7bd-d841-494f-b1c1-d182f7957ebb\") " pod="openshift-console/console-67547c88b6-pxvpl" Jul 09 04:33:30.863246 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:30.863179 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-oauth-serving-cert\") pod \"console-67547c88b6-pxvpl\" (UID: \"d2b8e7bd-d841-494f-b1c1-d182f7957ebb\") " pod="openshift-console/console-67547c88b6-pxvpl" Jul 09 04:33:30.863380 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:30.863360 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-trusted-ca-bundle\") pod \"console-67547c88b6-pxvpl\" (UID: \"d2b8e7bd-d841-494f-b1c1-d182f7957ebb\") " pod="openshift-console/console-67547c88b6-pxvpl" Jul 09 04:33:30.865062 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:30.865040 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-console-oauth-config\") pod \"console-67547c88b6-pxvpl\" (UID: \"d2b8e7bd-d841-494f-b1c1-d182f7957ebb\") " pod="openshift-console/console-67547c88b6-pxvpl" Jul 09 04:33:30.865262 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:30.865241 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-console-serving-cert\") pod \"console-67547c88b6-pxvpl\" (UID: \"d2b8e7bd-d841-494f-b1c1-d182f7957ebb\") " pod="openshift-console/console-67547c88b6-pxvpl" Jul 09 04:33:30.870388 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:30.870367 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mtjs\" (UniqueName: \"kubernetes.io/projected/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-kube-api-access-9mtjs\") pod \"console-67547c88b6-pxvpl\" (UID: \"d2b8e7bd-d841-494f-b1c1-d182f7957ebb\") " pod="openshift-console/console-67547c88b6-pxvpl" Jul 09 04:33:31.002317 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:31.002235 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67547c88b6-pxvpl" Jul 09 04:33:31.295589 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:31.294636 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:31.354830 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:31.354605 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67547c88b6-pxvpl"] Jul 09 04:33:31.357163 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:31.357089 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:31.364464 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:33:31.364360 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2b8e7bd_d841_494f_b1c1_d182f7957ebb.slice/crio-28b407186b7951090d216a5d81dad9dcc2caa38218bd6d770496c845620e76b7 WatchSource:0}: Error finding container 28b407186b7951090d216a5d81dad9dcc2caa38218bd6d770496c845620e76b7: Status 404 returned error can't find the container with id 28b407186b7951090d216a5d81dad9dcc2caa38218bd6d770496c845620e76b7 Jul 09 04:33:31.422439 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:31.422405 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:31.518882 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:31.518801 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:31.674479 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:31.674447 2565 generic.go:356] "Generic (PLEG): container finished" podID="3867ddde-c7e8-4e28-b44f-ce053ff6e7f9" containerID="76d0a4fcf52aa2d83b31e6ee23986e53da4132e6fea4e882ee0a54b9f94f73d0" exitCode=0 Jul 09 04:33:31.675124 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:31.674541 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6slcp" event={"ID":"3867ddde-c7e8-4e28-b44f-ce053ff6e7f9","Type":"ContainerDied","Data":"76d0a4fcf52aa2d83b31e6ee23986e53da4132e6fea4e882ee0a54b9f94f73d0"} Jul 09 04:33:31.676954 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:31.676917 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-mvq75" event={"ID":"ffccf1f2-c6d1-4d17-8387-a38651058f81","Type":"ContainerStarted","Data":"d8dc48fbfe9f800f0c0829007f2d7c0ef9992e4108a71a441b3812897a0d20c8"} Jul 09 04:33:31.677086 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:31.676977 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-mvq75" event={"ID":"ffccf1f2-c6d1-4d17-8387-a38651058f81","Type":"ContainerStarted","Data":"4ce81215b5671e611176be555eec6f6d0a75ded17fe1a73af41094b549464b0b"} Jul 09 04:33:31.677086 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:31.676999 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-mvq75" event={"ID":"ffccf1f2-c6d1-4d17-8387-a38651058f81","Type":"ContainerStarted","Data":"56ed57a6e7d594d2999f67294fe1ee18247205e351fd47151a3fb5381fc87719"} Jul 09 04:33:31.678293 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:31.678269 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67547c88b6-pxvpl" event={"ID":"d2b8e7bd-d841-494f-b1c1-d182f7957ebb","Type":"ContainerStarted","Data":"28b407186b7951090d216a5d81dad9dcc2caa38218bd6d770496c845620e76b7"} Jul 09 04:33:31.708713 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:31.708661 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-mvq75" podStartSLOduration=1.928338055 podStartE2EDuration="3.708643366s" totalImagesPullingTime="1.780305311s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:33:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:33:31.707271713 +0000 UTC m=+83.958795292" watchObservedRunningTime="2026-07-09 04:33:31.708643366 +0000 UTC m=+83.960166945" Jul 09 04:33:32.068016 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:32.067933 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:32.143476 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:32.143442 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:32.684407 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:32.684369 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6slcp" event={"ID":"3867ddde-c7e8-4e28-b44f-ce053ff6e7f9","Type":"ContainerStarted","Data":"4ec1c8f1c38b210e99182fc5d1b66b3f8e451d21ac4a2069089a3eee95ab6a37"} Jul 09 04:33:32.684883 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:32.684434 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6slcp" event={"ID":"3867ddde-c7e8-4e28-b44f-ce053ff6e7f9","Type":"ContainerStarted","Data":"02adbb16bba2013f73e227c0df4245927e988a7330ef3b38f2b39f46866a8bb5"} Jul 09 04:33:32.709231 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:32.709175 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6slcp" podStartSLOduration=2.701906626 podStartE2EDuration="4.709159595s" totalImagesPullingTime="1.913419244s" totalInitContainerRuntime="93.833725ms" isStatefulPod=true podCreationTimestamp="2026-07-09 04:33:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:33:32.708634938 +0000 UTC m=+84.960158530" watchObservedRunningTime="2026-07-09 04:33:32.709159595 +0000 UTC m=+84.960683187" Jul 09 04:33:33.152817 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:33.152734 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:33.206868 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:33.206823 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:33.581043 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:33.581016 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-76b76c5bc7-4r6xd" Jul 09 04:33:33.692664 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:33.692634 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:33.836262 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:33.836173 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:33.888713 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:33.888676 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:34.177580 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:34.177502 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:34.693500 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:34.693407 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67547c88b6-pxvpl" event={"ID":"d2b8e7bd-d841-494f-b1c1-d182f7957ebb","Type":"ContainerStarted","Data":"ce2a38f1c0f55f8c22d3b731ba5cb7df291f1f3402c68e27b79a2120f99375c2"} Jul 09 04:33:34.713615 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:34.713563 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-67547c88b6-pxvpl" podStartSLOduration=1.690685797 podStartE2EDuration="4.713545338s" totalImagesPullingTime="3.022859541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:33:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:33:34.713171417 +0000 UTC m=+86.964695001" watchObservedRunningTime="2026-07-09 04:33:34.713545338 +0000 UTC m=+86.965068916" Jul 09 04:33:35.083075 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:35.082982 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:35.133680 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:35.133644 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:35.336083 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:35.336008 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:35.393207 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:35.393180 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:35.813162 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:35.813130 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:35.901821 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:35.901784 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67547c88b6-pxvpl"] Jul 09 04:33:35.905524 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:35.905497 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:37.492100 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:37.492064 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:37.546793 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:37.546762 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:38.123856 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:38.123822 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:38.172277 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:38.172247 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:38.846613 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:38.846577 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:38.946659 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:38.946626 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:41.003093 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:41.003051 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-67547c88b6-pxvpl" Jul 09 04:33:41.115457 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:41.115424 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:41.215807 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:41.215776 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:44.902356 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:44.902307 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:44.963470 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:44.963437 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:47.140959 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:47.140877 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:47.241880 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:47.241846 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:47.342560 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:47.342529 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:50.340099 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:50.340054 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:50.439867 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:50.439837 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:33:51.624801 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:51.624773 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-bgsl7" Jul 09 04:33:57.102301 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:33:57.102269 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wbngq_964ab4da-c25d-46bd-97df-ffb2cd5b820b/serve-healthcheck-canary/0.log" Jul 09 04:34:01.720612 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:01.720539 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-67547c88b6-pxvpl" podUID="d2b8e7bd-d841-494f-b1c1-d182f7957ebb" containerName="console" containerID="cri-o://ce2a38f1c0f55f8c22d3b731ba5cb7df291f1f3402c68e27b79a2120f99375c2" gracePeriod=15 Jul 09 04:34:01.963808 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:01.963782 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67547c88b6-pxvpl" Jul 09 04:34:02.006264 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:02.006169 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-service-ca\") pod \"d2b8e7bd-d841-494f-b1c1-d182f7957ebb\" (UID: \"d2b8e7bd-d841-494f-b1c1-d182f7957ebb\") " Jul 09 04:34:02.006264 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:02.006211 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-console-oauth-config\") pod \"d2b8e7bd-d841-494f-b1c1-d182f7957ebb\" (UID: \"d2b8e7bd-d841-494f-b1c1-d182f7957ebb\") " Jul 09 04:34:02.006264 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:02.006237 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-console-config\") pod \"d2b8e7bd-d841-494f-b1c1-d182f7957ebb\" (UID: \"d2b8e7bd-d841-494f-b1c1-d182f7957ebb\") " Jul 09 04:34:02.006673 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:02.006273 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-console-serving-cert\") pod \"d2b8e7bd-d841-494f-b1c1-d182f7957ebb\" (UID: \"d2b8e7bd-d841-494f-b1c1-d182f7957ebb\") " Jul 09 04:34:02.006673 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:02.006299 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9mtjs\" (UniqueName: \"kubernetes.io/projected/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-kube-api-access-9mtjs\") pod \"d2b8e7bd-d841-494f-b1c1-d182f7957ebb\" (UID: \"d2b8e7bd-d841-494f-b1c1-d182f7957ebb\") " Jul 09 04:34:02.006673 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:02.006315 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-trusted-ca-bundle\") pod \"d2b8e7bd-d841-494f-b1c1-d182f7957ebb\" (UID: \"d2b8e7bd-d841-494f-b1c1-d182f7957ebb\") " Jul 09 04:34:02.006673 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:02.006395 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-oauth-serving-cert\") pod \"d2b8e7bd-d841-494f-b1c1-d182f7957ebb\" (UID: \"d2b8e7bd-d841-494f-b1c1-d182f7957ebb\") " Jul 09 04:34:02.006894 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:02.006786 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-console-config" (OuterVolumeSpecName: "console-config") pod "d2b8e7bd-d841-494f-b1c1-d182f7957ebb" (UID: "d2b8e7bd-d841-494f-b1c1-d182f7957ebb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:34:02.006894 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:02.006820 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-service-ca" (OuterVolumeSpecName: "service-ca") pod "d2b8e7bd-d841-494f-b1c1-d182f7957ebb" (UID: "d2b8e7bd-d841-494f-b1c1-d182f7957ebb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:34:02.006997 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:02.006931 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d2b8e7bd-d841-494f-b1c1-d182f7957ebb" (UID: "d2b8e7bd-d841-494f-b1c1-d182f7957ebb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:34:02.006997 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:02.006979 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d2b8e7bd-d841-494f-b1c1-d182f7957ebb" (UID: "d2b8e7bd-d841-494f-b1c1-d182f7957ebb"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:34:02.008723 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:02.008692 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d2b8e7bd-d841-494f-b1c1-d182f7957ebb" (UID: "d2b8e7bd-d841-494f-b1c1-d182f7957ebb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:34:02.008723 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:02.008717 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-kube-api-access-9mtjs" (OuterVolumeSpecName: "kube-api-access-9mtjs") pod "d2b8e7bd-d841-494f-b1c1-d182f7957ebb" (UID: "d2b8e7bd-d841-494f-b1c1-d182f7957ebb"). InnerVolumeSpecName "kube-api-access-9mtjs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:34:02.008871 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:02.008744 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d2b8e7bd-d841-494f-b1c1-d182f7957ebb" (UID: "d2b8e7bd-d841-494f-b1c1-d182f7957ebb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:34:02.107512 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:02.107465 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9mtjs\" (UniqueName: \"kubernetes.io/projected/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-kube-api-access-9mtjs\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:34:02.107512 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:02.107503 2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-trusted-ca-bundle\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:34:02.107512 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:02.107514 2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-oauth-serving-cert\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:34:02.107512 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:02.107525 2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-service-ca\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:34:02.107786 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:02.107534 2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-console-oauth-config\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:34:02.107786 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:02.107543 2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-console-config\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:34:02.107786 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:02.107551 2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d2b8e7bd-d841-494f-b1c1-d182f7957ebb-console-serving-cert\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:34:02.787455 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:02.787419 2565 generic.go:356] "Generic (PLEG): container finished" podID="d2b8e7bd-d841-494f-b1c1-d182f7957ebb" containerID="ce2a38f1c0f55f8c22d3b731ba5cb7df291f1f3402c68e27b79a2120f99375c2" exitCode=0 Jul 09 04:34:02.787901 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:02.787464 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67547c88b6-pxvpl" Jul 09 04:34:02.787901 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:02.787479 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67547c88b6-pxvpl" event={"ID":"d2b8e7bd-d841-494f-b1c1-d182f7957ebb","Type":"ContainerDied","Data":"ce2a38f1c0f55f8c22d3b731ba5cb7df291f1f3402c68e27b79a2120f99375c2"} Jul 09 04:34:02.787901 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:02.787520 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67547c88b6-pxvpl" event={"ID":"d2b8e7bd-d841-494f-b1c1-d182f7957ebb","Type":"ContainerDied","Data":"28b407186b7951090d216a5d81dad9dcc2caa38218bd6d770496c845620e76b7"} Jul 09 04:34:02.787901 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:02.787552 2565 scope.go:117] "RemoveContainer" containerID="ce2a38f1c0f55f8c22d3b731ba5cb7df291f1f3402c68e27b79a2120f99375c2" Jul 09 04:34:02.795389 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:02.795365 2565 scope.go:117] "RemoveContainer" containerID="ce2a38f1c0f55f8c22d3b731ba5cb7df291f1f3402c68e27b79a2120f99375c2" Jul 09 04:34:02.795711 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:34:02.795685 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce2a38f1c0f55f8c22d3b731ba5cb7df291f1f3402c68e27b79a2120f99375c2\": container with ID starting with ce2a38f1c0f55f8c22d3b731ba5cb7df291f1f3402c68e27b79a2120f99375c2 not found: ID does not exist" containerID="ce2a38f1c0f55f8c22d3b731ba5cb7df291f1f3402c68e27b79a2120f99375c2" Jul 09 04:34:02.795783 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:02.795719 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce2a38f1c0f55f8c22d3b731ba5cb7df291f1f3402c68e27b79a2120f99375c2"} err="failed to get container status \"ce2a38f1c0f55f8c22d3b731ba5cb7df291f1f3402c68e27b79a2120f99375c2\": rpc error: code = NotFound desc = could not find container \"ce2a38f1c0f55f8c22d3b731ba5cb7df291f1f3402c68e27b79a2120f99375c2\": container with ID starting with ce2a38f1c0f55f8c22d3b731ba5cb7df291f1f3402c68e27b79a2120f99375c2 not found: ID does not exist" Jul 09 04:34:02.804131 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:02.804107 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67547c88b6-pxvpl"] Jul 09 04:34:02.807715 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:02.807695 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-67547c88b6-pxvpl"] Jul 09 04:34:04.194522 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:04.194491 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2b8e7bd-d841-494f-b1c1-d182f7957ebb" path="/var/lib/kubelet/pods/d2b8e7bd-d841-494f-b1c1-d182f7957ebb/volumes" Jul 09 04:34:05.355217 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:05.355185 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:34:05.406651 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:05.406618 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:34:10.280437 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:10.280398 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:34:10.380604 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:10.380574 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:34:10.444574 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:10.444539 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:34:47.594748 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:47.594707 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:34:47.865087 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:47.865010 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:34:47.917965 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:47.917936 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:34:56.188837 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.188803 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c66c869f6-r2lns"] Jul 09 04:34:56.189242 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.189047 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2b8e7bd-d841-494f-b1c1-d182f7957ebb" containerName="console" Jul 09 04:34:56.189242 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.189057 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2b8e7bd-d841-494f-b1c1-d182f7957ebb" containerName="console" Jul 09 04:34:56.189242 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.189097 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2b8e7bd-d841-494f-b1c1-d182f7957ebb" containerName="console" Jul 09 04:34:56.192197 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.192173 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c66c869f6-r2lns" Jul 09 04:34:56.195017 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.194996 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 09 04:34:56.195017 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.195012 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 09 04:34:56.195209 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.195029 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 09 04:34:56.196104 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.196086 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 09 04:34:56.196312 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.196291 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 09 04:34:56.196312 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.196299 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 09 04:34:56.196601 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.196586 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-k7dhp\"" Jul 09 04:34:56.196709 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.196692 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 09 04:34:56.201405 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.201386 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 09 04:34:56.202450 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.202426 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c66c869f6-r2lns"] Jul 09 04:34:56.271745 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.271705 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:34:56.310748 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.310713 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5393dcac-2b2f-4366-aae2-9700f5949c9f-console-oauth-config\") pod \"console-c66c869f6-r2lns\" (UID: \"5393dcac-2b2f-4366-aae2-9700f5949c9f\") " pod="openshift-console/console-c66c869f6-r2lns" Jul 09 04:34:56.310929 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.310772 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5393dcac-2b2f-4366-aae2-9700f5949c9f-trusted-ca-bundle\") pod \"console-c66c869f6-r2lns\" (UID: \"5393dcac-2b2f-4366-aae2-9700f5949c9f\") " pod="openshift-console/console-c66c869f6-r2lns" Jul 09 04:34:56.310929 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.310798 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5393dcac-2b2f-4366-aae2-9700f5949c9f-console-serving-cert\") pod \"console-c66c869f6-r2lns\" (UID: \"5393dcac-2b2f-4366-aae2-9700f5949c9f\") " pod="openshift-console/console-c66c869f6-r2lns" Jul 09 04:34:56.311063 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.310925 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5393dcac-2b2f-4366-aae2-9700f5949c9f-console-config\") pod \"console-c66c869f6-r2lns\" (UID: \"5393dcac-2b2f-4366-aae2-9700f5949c9f\") " pod="openshift-console/console-c66c869f6-r2lns" Jul 09 04:34:56.311063 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.310961 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5393dcac-2b2f-4366-aae2-9700f5949c9f-service-ca\") pod \"console-c66c869f6-r2lns\" (UID: \"5393dcac-2b2f-4366-aae2-9700f5949c9f\") " pod="openshift-console/console-c66c869f6-r2lns" Jul 09 04:34:56.311063 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.311002 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pt9g\" (UniqueName: \"kubernetes.io/projected/5393dcac-2b2f-4366-aae2-9700f5949c9f-kube-api-access-2pt9g\") pod \"console-c66c869f6-r2lns\" (UID: \"5393dcac-2b2f-4366-aae2-9700f5949c9f\") " pod="openshift-console/console-c66c869f6-r2lns" Jul 09 04:34:56.311167 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.311060 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5393dcac-2b2f-4366-aae2-9700f5949c9f-oauth-serving-cert\") pod \"console-c66c869f6-r2lns\" (UID: \"5393dcac-2b2f-4366-aae2-9700f5949c9f\") " pod="openshift-console/console-c66c869f6-r2lns" Jul 09 04:34:56.324606 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.324570 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:34:56.411889 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.411854 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5393dcac-2b2f-4366-aae2-9700f5949c9f-console-oauth-config\") pod \"console-c66c869f6-r2lns\" (UID: \"5393dcac-2b2f-4366-aae2-9700f5949c9f\") " pod="openshift-console/console-c66c869f6-r2lns" Jul 09 04:34:56.412049 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.411903 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5393dcac-2b2f-4366-aae2-9700f5949c9f-trusted-ca-bundle\") pod \"console-c66c869f6-r2lns\" (UID: \"5393dcac-2b2f-4366-aae2-9700f5949c9f\") " pod="openshift-console/console-c66c869f6-r2lns" Jul 09 04:34:56.412049 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.411934 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5393dcac-2b2f-4366-aae2-9700f5949c9f-console-serving-cert\") pod \"console-c66c869f6-r2lns\" (UID: \"5393dcac-2b2f-4366-aae2-9700f5949c9f\") " pod="openshift-console/console-c66c869f6-r2lns" Jul 09 04:34:56.412049 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.411971 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5393dcac-2b2f-4366-aae2-9700f5949c9f-console-config\") pod \"console-c66c869f6-r2lns\" (UID: \"5393dcac-2b2f-4366-aae2-9700f5949c9f\") " pod="openshift-console/console-c66c869f6-r2lns" Jul 09 04:34:56.412049 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.411993 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5393dcac-2b2f-4366-aae2-9700f5949c9f-service-ca\") pod \"console-c66c869f6-r2lns\" (UID: \"5393dcac-2b2f-4366-aae2-9700f5949c9f\") " pod="openshift-console/console-c66c869f6-r2lns" Jul 09 04:34:56.412049 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.412018 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pt9g\" (UniqueName: \"kubernetes.io/projected/5393dcac-2b2f-4366-aae2-9700f5949c9f-kube-api-access-2pt9g\") pod \"console-c66c869f6-r2lns\" (UID: \"5393dcac-2b2f-4366-aae2-9700f5949c9f\") " pod="openshift-console/console-c66c869f6-r2lns" Jul 09 04:34:56.412285 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.412122 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5393dcac-2b2f-4366-aae2-9700f5949c9f-oauth-serving-cert\") pod \"console-c66c869f6-r2lns\" (UID: \"5393dcac-2b2f-4366-aae2-9700f5949c9f\") " pod="openshift-console/console-c66c869f6-r2lns" Jul 09 04:34:56.412831 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.412806 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5393dcac-2b2f-4366-aae2-9700f5949c9f-oauth-serving-cert\") pod \"console-c66c869f6-r2lns\" (UID: \"5393dcac-2b2f-4366-aae2-9700f5949c9f\") " pod="openshift-console/console-c66c869f6-r2lns" Jul 09 04:34:56.412951 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.412837 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5393dcac-2b2f-4366-aae2-9700f5949c9f-console-config\") pod \"console-c66c869f6-r2lns\" (UID: \"5393dcac-2b2f-4366-aae2-9700f5949c9f\") " pod="openshift-console/console-c66c869f6-r2lns" Jul 09 04:34:56.412951 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.412881 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5393dcac-2b2f-4366-aae2-9700f5949c9f-service-ca\") pod \"console-c66c869f6-r2lns\" (UID: \"5393dcac-2b2f-4366-aae2-9700f5949c9f\") " pod="openshift-console/console-c66c869f6-r2lns" Jul 09 04:34:56.413072 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.413020 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5393dcac-2b2f-4366-aae2-9700f5949c9f-trusted-ca-bundle\") pod \"console-c66c869f6-r2lns\" (UID: \"5393dcac-2b2f-4366-aae2-9700f5949c9f\") " pod="openshift-console/console-c66c869f6-r2lns" Jul 09 04:34:56.414387 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.414361 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5393dcac-2b2f-4366-aae2-9700f5949c9f-console-oauth-config\") pod \"console-c66c869f6-r2lns\" (UID: \"5393dcac-2b2f-4366-aae2-9700f5949c9f\") " pod="openshift-console/console-c66c869f6-r2lns" Jul 09 04:34:56.414463 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.414455 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5393dcac-2b2f-4366-aae2-9700f5949c9f-console-serving-cert\") pod \"console-c66c869f6-r2lns\" (UID: \"5393dcac-2b2f-4366-aae2-9700f5949c9f\") " pod="openshift-console/console-c66c869f6-r2lns" Jul 09 04:34:56.421177 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.421152 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pt9g\" (UniqueName: \"kubernetes.io/projected/5393dcac-2b2f-4366-aae2-9700f5949c9f-kube-api-access-2pt9g\") pod \"console-c66c869f6-r2lns\" (UID: \"5393dcac-2b2f-4366-aae2-9700f5949c9f\") " pod="openshift-console/console-c66c869f6-r2lns" Jul 09 04:34:56.502164 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.502075 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c66c869f6-r2lns" Jul 09 04:34:56.647704 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.647679 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c66c869f6-r2lns"] Jul 09 04:34:56.650125 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:34:56.650098 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5393dcac_2b2f_4366_aae2_9700f5949c9f.slice/crio-d10e127d36b9089904078894aefc1841cfaa80dff46665ee46773cdefa6c8b61 WatchSource:0}: Error finding container d10e127d36b9089904078894aefc1841cfaa80dff46665ee46773cdefa6c8b61: Status 404 returned error can't find the container with id d10e127d36b9089904078894aefc1841cfaa80dff46665ee46773cdefa6c8b61 Jul 09 04:34:56.967515 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.967482 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c66c869f6-r2lns" event={"ID":"5393dcac-2b2f-4366-aae2-9700f5949c9f","Type":"ContainerStarted","Data":"950f8e43ad3dab875d64773d1254e6ddd292431d4622440bd4eb0080a8c75575"} Jul 09 04:34:56.967706 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.967532 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c66c869f6-r2lns" event={"ID":"5393dcac-2b2f-4366-aae2-9700f5949c9f","Type":"ContainerStarted","Data":"d10e127d36b9089904078894aefc1841cfaa80dff46665ee46773cdefa6c8b61"} Jul 09 04:34:56.984281 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:34:56.984234 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c66c869f6-r2lns" podStartSLOduration=0.984217022 podStartE2EDuration="984.217022ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:34:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:34:56.983331173 +0000 UTC m=+169.234854754" watchObservedRunningTime="2026-07-09 04:34:56.984217022 +0000 UTC m=+169.235740599" Jul 09 04:35:06.503172 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:35:06.503078 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c66c869f6-r2lns" Jul 09 04:35:06.503172 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:35:06.503149 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-c66c869f6-r2lns" Jul 09 04:35:06.507895 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:35:06.507870 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-c66c869f6-r2lns" Jul 09 04:35:07.003267 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:35:07.003242 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-c66c869f6-r2lns" Jul 09 04:35:07.144505 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:35:07.144471 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:35:07.312996 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:35:07.312919 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:35:07.361684 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:35:07.361657 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:35:20.900365 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:35:20.900310 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:35:27.211467 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:35:27.211435 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:35:27.263980 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:35:27.263946 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:36:02.825290 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:02.825251 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:36:02.872039 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:02.872014 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:36:04.241558 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:04.241526 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c845bc447-d42qw"] Jul 09 04:36:04.244505 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:04.244487 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c845bc447-d42qw" Jul 09 04:36:04.262606 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:04.262583 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c845bc447-d42qw"] Jul 09 04:36:04.313558 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:04.313524 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/680a9c76-039a-4113-bd5e-ff063c22bcdf-console-serving-cert\") pod \"console-6c845bc447-d42qw\" (UID: \"680a9c76-039a-4113-bd5e-ff063c22bcdf\") " pod="openshift-console/console-6c845bc447-d42qw" Jul 09 04:36:04.313558 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:04.313558 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/680a9c76-039a-4113-bd5e-ff063c22bcdf-trusted-ca-bundle\") pod \"console-6c845bc447-d42qw\" (UID: \"680a9c76-039a-4113-bd5e-ff063c22bcdf\") " pod="openshift-console/console-6c845bc447-d42qw" Jul 09 04:36:04.313768 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:04.313582 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg2xq\" (UniqueName: \"kubernetes.io/projected/680a9c76-039a-4113-bd5e-ff063c22bcdf-kube-api-access-tg2xq\") pod \"console-6c845bc447-d42qw\" (UID: \"680a9c76-039a-4113-bd5e-ff063c22bcdf\") " pod="openshift-console/console-6c845bc447-d42qw" Jul 09 04:36:04.313768 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:04.313629 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/680a9c76-039a-4113-bd5e-ff063c22bcdf-console-config\") pod \"console-6c845bc447-d42qw\" (UID: \"680a9c76-039a-4113-bd5e-ff063c22bcdf\") " pod="openshift-console/console-6c845bc447-d42qw" Jul 09 04:36:04.313768 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:04.313681 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/680a9c76-039a-4113-bd5e-ff063c22bcdf-console-oauth-config\") pod \"console-6c845bc447-d42qw\" (UID: \"680a9c76-039a-4113-bd5e-ff063c22bcdf\") " pod="openshift-console/console-6c845bc447-d42qw" Jul 09 04:36:04.313768 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:04.313733 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/680a9c76-039a-4113-bd5e-ff063c22bcdf-oauth-serving-cert\") pod \"console-6c845bc447-d42qw\" (UID: \"680a9c76-039a-4113-bd5e-ff063c22bcdf\") " pod="openshift-console/console-6c845bc447-d42qw" Jul 09 04:36:04.313768 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:04.313751 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/680a9c76-039a-4113-bd5e-ff063c22bcdf-service-ca\") pod \"console-6c845bc447-d42qw\" (UID: \"680a9c76-039a-4113-bd5e-ff063c22bcdf\") " pod="openshift-console/console-6c845bc447-d42qw" Jul 09 04:36:04.325988 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:04.325954 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:36:04.414915 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:04.414878 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/680a9c76-039a-4113-bd5e-ff063c22bcdf-console-serving-cert\") pod \"console-6c845bc447-d42qw\" (UID: \"680a9c76-039a-4113-bd5e-ff063c22bcdf\") " pod="openshift-console/console-6c845bc447-d42qw" Jul 09 04:36:04.414915 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:04.414916 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/680a9c76-039a-4113-bd5e-ff063c22bcdf-trusted-ca-bundle\") pod \"console-6c845bc447-d42qw\" (UID: \"680a9c76-039a-4113-bd5e-ff063c22bcdf\") " pod="openshift-console/console-6c845bc447-d42qw" Jul 09 04:36:04.415149 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:04.414945 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg2xq\" (UniqueName: \"kubernetes.io/projected/680a9c76-039a-4113-bd5e-ff063c22bcdf-kube-api-access-tg2xq\") pod \"console-6c845bc447-d42qw\" (UID: \"680a9c76-039a-4113-bd5e-ff063c22bcdf\") " pod="openshift-console/console-6c845bc447-d42qw" Jul 09 04:36:04.415149 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:04.415080 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/680a9c76-039a-4113-bd5e-ff063c22bcdf-console-config\") pod \"console-6c845bc447-d42qw\" (UID: \"680a9c76-039a-4113-bd5e-ff063c22bcdf\") " pod="openshift-console/console-6c845bc447-d42qw" Jul 09 04:36:04.415250 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:04.415180 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/680a9c76-039a-4113-bd5e-ff063c22bcdf-console-oauth-config\") pod \"console-6c845bc447-d42qw\" (UID: \"680a9c76-039a-4113-bd5e-ff063c22bcdf\") " pod="openshift-console/console-6c845bc447-d42qw" Jul 09 04:36:04.415250 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:04.415233 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/680a9c76-039a-4113-bd5e-ff063c22bcdf-oauth-serving-cert\") pod \"console-6c845bc447-d42qw\" (UID: \"680a9c76-039a-4113-bd5e-ff063c22bcdf\") " pod="openshift-console/console-6c845bc447-d42qw" Jul 09 04:36:04.415384 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:04.415261 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/680a9c76-039a-4113-bd5e-ff063c22bcdf-service-ca\") pod \"console-6c845bc447-d42qw\" (UID: \"680a9c76-039a-4113-bd5e-ff063c22bcdf\") " pod="openshift-console/console-6c845bc447-d42qw" Jul 09 04:36:04.415848 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:04.415822 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/680a9c76-039a-4113-bd5e-ff063c22bcdf-console-config\") pod \"console-6c845bc447-d42qw\" (UID: \"680a9c76-039a-4113-bd5e-ff063c22bcdf\") " pod="openshift-console/console-6c845bc447-d42qw" Jul 09 04:36:04.415943 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:04.415902 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/680a9c76-039a-4113-bd5e-ff063c22bcdf-trusted-ca-bundle\") pod \"console-6c845bc447-d42qw\" (UID: \"680a9c76-039a-4113-bd5e-ff063c22bcdf\") " pod="openshift-console/console-6c845bc447-d42qw" Jul 09 04:36:04.415943 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:04.415920 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/680a9c76-039a-4113-bd5e-ff063c22bcdf-oauth-serving-cert\") pod \"console-6c845bc447-d42qw\" (UID: \"680a9c76-039a-4113-bd5e-ff063c22bcdf\") " pod="openshift-console/console-6c845bc447-d42qw" Jul 09 04:36:04.415943 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:04.415932 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/680a9c76-039a-4113-bd5e-ff063c22bcdf-service-ca\") pod \"console-6c845bc447-d42qw\" (UID: \"680a9c76-039a-4113-bd5e-ff063c22bcdf\") " pod="openshift-console/console-6c845bc447-d42qw" Jul 09 04:36:04.417408 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:04.417385 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/680a9c76-039a-4113-bd5e-ff063c22bcdf-console-oauth-config\") pod \"console-6c845bc447-d42qw\" (UID: \"680a9c76-039a-4113-bd5e-ff063c22bcdf\") " pod="openshift-console/console-6c845bc447-d42qw" Jul 09 04:36:04.417524 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:04.417507 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/680a9c76-039a-4113-bd5e-ff063c22bcdf-console-serving-cert\") pod \"console-6c845bc447-d42qw\" (UID: \"680a9c76-039a-4113-bd5e-ff063c22bcdf\") " pod="openshift-console/console-6c845bc447-d42qw" Jul 09 04:36:04.421786 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:04.421767 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg2xq\" (UniqueName: \"kubernetes.io/projected/680a9c76-039a-4113-bd5e-ff063c22bcdf-kube-api-access-tg2xq\") pod \"console-6c845bc447-d42qw\" (UID: \"680a9c76-039a-4113-bd5e-ff063c22bcdf\") " pod="openshift-console/console-6c845bc447-d42qw" Jul 09 04:36:04.425776 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:04.425754 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:36:04.472734 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:04.472705 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:36:04.553656 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:04.553565 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c845bc447-d42qw" Jul 09 04:36:04.679275 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:04.679251 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c845bc447-d42qw"] Jul 09 04:36:04.681653 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:36:04.681630 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod680a9c76_039a_4113_bd5e_ff063c22bcdf.slice/crio-39fc63dae10cb53d8fcc7f7baf124c53698ca9601fb1737d27c4ac93d432a961 WatchSource:0}: Error finding container 39fc63dae10cb53d8fcc7f7baf124c53698ca9601fb1737d27c4ac93d432a961: Status 404 returned error can't find the container with id 39fc63dae10cb53d8fcc7f7baf124c53698ca9601fb1737d27c4ac93d432a961 Jul 09 04:36:05.040705 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:05.040674 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:36:05.091705 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:05.091677 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:36:05.200785 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:05.200752 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c845bc447-d42qw" event={"ID":"680a9c76-039a-4113-bd5e-ff063c22bcdf","Type":"ContainerStarted","Data":"608d9ba7bcbafe28adead8db7033e531241f4268ad3b026a45279b97685fb61d"} Jul 09 04:36:05.200976 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:05.200797 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c845bc447-d42qw" event={"ID":"680a9c76-039a-4113-bd5e-ff063c22bcdf","Type":"ContainerStarted","Data":"39fc63dae10cb53d8fcc7f7baf124c53698ca9601fb1737d27c4ac93d432a961"} Jul 09 04:36:05.217317 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:05.217270 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c845bc447-d42qw" podStartSLOduration=1.217254967 podStartE2EDuration="1.217254967s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:36:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:36:05.216505739 +0000 UTC m=+237.468029317" watchObservedRunningTime="2026-07-09 04:36:05.217254967 +0000 UTC m=+237.468778546" Jul 09 04:36:12.226969 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:12.226928 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:36:14.554266 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:14.554225 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c845bc447-d42qw" Jul 09 04:36:14.554673 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:14.554290 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c845bc447-d42qw" Jul 09 04:36:14.559079 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:14.559058 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c845bc447-d42qw" Jul 09 04:36:14.827763 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:14.827676 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:36:14.874978 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:14.874934 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:36:15.239770 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:15.239738 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c845bc447-d42qw" Jul 09 04:36:15.281954 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:15.281921 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c66c869f6-r2lns"] Jul 09 04:36:15.391979 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:15.391945 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:36:15.490638 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:15.490565 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:36:20.353467 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:20.353435 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:36:20.454982 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:20.454941 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:36:40.302739 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:40.302634 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-c66c869f6-r2lns" podUID="5393dcac-2b2f-4366-aae2-9700f5949c9f" containerName="console" containerID="cri-o://950f8e43ad3dab875d64773d1254e6ddd292431d4622440bd4eb0080a8c75575" gracePeriod=15 Jul 09 04:36:40.537902 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:40.537879 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c66c869f6-r2lns" Jul 09 04:36:40.594408 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:40.594305 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5393dcac-2b2f-4366-aae2-9700f5949c9f-console-serving-cert\") pod \"5393dcac-2b2f-4366-aae2-9700f5949c9f\" (UID: \"5393dcac-2b2f-4366-aae2-9700f5949c9f\") " Jul 09 04:36:40.594408 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:40.594364 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5393dcac-2b2f-4366-aae2-9700f5949c9f-oauth-serving-cert\") pod \"5393dcac-2b2f-4366-aae2-9700f5949c9f\" (UID: \"5393dcac-2b2f-4366-aae2-9700f5949c9f\") " Jul 09 04:36:40.594582 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:40.594454 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5393dcac-2b2f-4366-aae2-9700f5949c9f-console-config\") pod \"5393dcac-2b2f-4366-aae2-9700f5949c9f\" (UID: \"5393dcac-2b2f-4366-aae2-9700f5949c9f\") " Jul 09 04:36:40.594582 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:40.594485 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5393dcac-2b2f-4366-aae2-9700f5949c9f-trusted-ca-bundle\") pod \"5393dcac-2b2f-4366-aae2-9700f5949c9f\" (UID: \"5393dcac-2b2f-4366-aae2-9700f5949c9f\") " Jul 09 04:36:40.594582 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:40.594526 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5393dcac-2b2f-4366-aae2-9700f5949c9f-service-ca\") pod \"5393dcac-2b2f-4366-aae2-9700f5949c9f\" (UID: \"5393dcac-2b2f-4366-aae2-9700f5949c9f\") " Jul 09 04:36:40.594582 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:40.594572 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2pt9g\" (UniqueName: \"kubernetes.io/projected/5393dcac-2b2f-4366-aae2-9700f5949c9f-kube-api-access-2pt9g\") pod \"5393dcac-2b2f-4366-aae2-9700f5949c9f\" (UID: \"5393dcac-2b2f-4366-aae2-9700f5949c9f\") " Jul 09 04:36:40.594777 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:40.594622 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5393dcac-2b2f-4366-aae2-9700f5949c9f-console-oauth-config\") pod \"5393dcac-2b2f-4366-aae2-9700f5949c9f\" (UID: \"5393dcac-2b2f-4366-aae2-9700f5949c9f\") " Jul 09 04:36:40.594831 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:40.594793 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5393dcac-2b2f-4366-aae2-9700f5949c9f-console-config" (OuterVolumeSpecName: "console-config") pod "5393dcac-2b2f-4366-aae2-9700f5949c9f" (UID: "5393dcac-2b2f-4366-aae2-9700f5949c9f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:36:40.594831 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:40.594732 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5393dcac-2b2f-4366-aae2-9700f5949c9f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5393dcac-2b2f-4366-aae2-9700f5949c9f" (UID: "5393dcac-2b2f-4366-aae2-9700f5949c9f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:36:40.594932 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:40.594914 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5393dcac-2b2f-4366-aae2-9700f5949c9f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5393dcac-2b2f-4366-aae2-9700f5949c9f" (UID: "5393dcac-2b2f-4366-aae2-9700f5949c9f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:36:40.595015 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:40.594985 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5393dcac-2b2f-4366-aae2-9700f5949c9f-service-ca" (OuterVolumeSpecName: "service-ca") pod "5393dcac-2b2f-4366-aae2-9700f5949c9f" (UID: "5393dcac-2b2f-4366-aae2-9700f5949c9f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:36:40.596472 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:40.596451 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5393dcac-2b2f-4366-aae2-9700f5949c9f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5393dcac-2b2f-4366-aae2-9700f5949c9f" (UID: "5393dcac-2b2f-4366-aae2-9700f5949c9f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:36:40.596571 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:40.596556 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5393dcac-2b2f-4366-aae2-9700f5949c9f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5393dcac-2b2f-4366-aae2-9700f5949c9f" (UID: "5393dcac-2b2f-4366-aae2-9700f5949c9f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:36:40.596628 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:40.596592 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5393dcac-2b2f-4366-aae2-9700f5949c9f-kube-api-access-2pt9g" (OuterVolumeSpecName: "kube-api-access-2pt9g") pod "5393dcac-2b2f-4366-aae2-9700f5949c9f" (UID: "5393dcac-2b2f-4366-aae2-9700f5949c9f"). InnerVolumeSpecName "kube-api-access-2pt9g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:36:40.696156 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:40.696121 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2pt9g\" (UniqueName: \"kubernetes.io/projected/5393dcac-2b2f-4366-aae2-9700f5949c9f-kube-api-access-2pt9g\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:36:40.696156 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:40.696151 2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5393dcac-2b2f-4366-aae2-9700f5949c9f-console-oauth-config\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:36:40.696156 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:40.696161 2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5393dcac-2b2f-4366-aae2-9700f5949c9f-console-serving-cert\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:36:40.696423 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:40.696170 2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5393dcac-2b2f-4366-aae2-9700f5949c9f-oauth-serving-cert\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:36:40.696423 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:40.696179 2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5393dcac-2b2f-4366-aae2-9700f5949c9f-console-config\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:36:40.696423 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:40.696187 2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5393dcac-2b2f-4366-aae2-9700f5949c9f-trusted-ca-bundle\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:36:40.696423 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:40.696195 2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5393dcac-2b2f-4366-aae2-9700f5949c9f-service-ca\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:36:41.321790 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:41.321757 2565 generic.go:356] "Generic (PLEG): container finished" podID="5393dcac-2b2f-4366-aae2-9700f5949c9f" containerID="950f8e43ad3dab875d64773d1254e6ddd292431d4622440bd4eb0080a8c75575" exitCode=0 Jul 09 04:36:41.322212 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:41.321820 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c66c869f6-r2lns" Jul 09 04:36:41.322212 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:41.321829 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c66c869f6-r2lns" event={"ID":"5393dcac-2b2f-4366-aae2-9700f5949c9f","Type":"ContainerDied","Data":"950f8e43ad3dab875d64773d1254e6ddd292431d4622440bd4eb0080a8c75575"} Jul 09 04:36:41.322212 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:41.321928 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c66c869f6-r2lns" event={"ID":"5393dcac-2b2f-4366-aae2-9700f5949c9f","Type":"ContainerDied","Data":"d10e127d36b9089904078894aefc1841cfaa80dff46665ee46773cdefa6c8b61"} Jul 09 04:36:41.322212 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:41.321948 2565 scope.go:117] "RemoveContainer" containerID="950f8e43ad3dab875d64773d1254e6ddd292431d4622440bd4eb0080a8c75575" Jul 09 04:36:41.330250 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:41.330231 2565 scope.go:117] "RemoveContainer" containerID="950f8e43ad3dab875d64773d1254e6ddd292431d4622440bd4eb0080a8c75575" Jul 09 04:36:41.330518 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:36:41.330499 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"950f8e43ad3dab875d64773d1254e6ddd292431d4622440bd4eb0080a8c75575\": container with ID starting with 950f8e43ad3dab875d64773d1254e6ddd292431d4622440bd4eb0080a8c75575 not found: ID does not exist" containerID="950f8e43ad3dab875d64773d1254e6ddd292431d4622440bd4eb0080a8c75575" Jul 09 04:36:41.330583 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:41.330526 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"950f8e43ad3dab875d64773d1254e6ddd292431d4622440bd4eb0080a8c75575"} err="failed to get container status \"950f8e43ad3dab875d64773d1254e6ddd292431d4622440bd4eb0080a8c75575\": rpc error: code = NotFound desc = could not find container \"950f8e43ad3dab875d64773d1254e6ddd292431d4622440bd4eb0080a8c75575\": container with ID starting with 950f8e43ad3dab875d64773d1254e6ddd292431d4622440bd4eb0080a8c75575 not found: ID does not exist" Jul 09 04:36:41.340941 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:41.340921 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c66c869f6-r2lns"] Jul 09 04:36:41.344320 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:41.344295 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-c66c869f6-r2lns"] Jul 09 04:36:42.193881 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:42.193837 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5393dcac-2b2f-4366-aae2-9700f5949c9f" path="/var/lib/kubelet/pods/5393dcac-2b2f-4366-aae2-9700f5949c9f/volumes" Jul 09 04:36:58.066944 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:58.066910 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:36:58.166310 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:36:58.166237 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:37:08.099322 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:08.099295 2565 kubelet.go:1648] "Image garbage collection succeeded" Jul 09 04:37:24.669957 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:24.669928 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:37:26.264545 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:26.264502 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:37:28.275616 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:28.275580 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:37:28.375739 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:28.375703 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:37:29.439534 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:29.439502 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:37:29.580947 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:29.580913 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:37:30.384532 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:30.384498 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:37:30.559950 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:30.559915 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:37:32.512031 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:32.512002 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:37:32.936878 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:32.936846 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:37:32.993221 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:32.993190 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-622wz"] Jul 09 04:37:32.993469 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:32.993456 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5393dcac-2b2f-4366-aae2-9700f5949c9f" containerName="console" Jul 09 04:37:32.993469 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:32.993470 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5393dcac-2b2f-4366-aae2-9700f5949c9f" containerName="console" Jul 09 04:37:32.993586 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:32.993513 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5393dcac-2b2f-4366-aae2-9700f5949c9f" containerName="console" Jul 09 04:37:32.996290 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:32.996273 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-622wz" Jul 09 04:37:32.998680 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:32.998664 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 09 04:37:32.998759 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:32.998722 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 09 04:37:32.999606 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:32.999587 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-4zvrq\"" Jul 09 04:37:33.003542 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:33.003520 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-622wz"] Jul 09 04:37:33.086394 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:33.086364 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1a2ca818-6395-47ff-a251-e646566f573f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-622wz\" (UID: \"1a2ca818-6395-47ff-a251-e646566f573f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-622wz" Jul 09 04:37:33.086571 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:33.086429 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7dn7\" (UniqueName: \"kubernetes.io/projected/1a2ca818-6395-47ff-a251-e646566f573f-kube-api-access-x7dn7\") pod \"cert-manager-cainjector-559cdd8c87-622wz\" (UID: \"1a2ca818-6395-47ff-a251-e646566f573f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-622wz" Jul 09 04:37:33.187252 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:33.187174 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1a2ca818-6395-47ff-a251-e646566f573f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-622wz\" (UID: \"1a2ca818-6395-47ff-a251-e646566f573f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-622wz" Jul 09 04:37:33.187407 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:33.187358 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7dn7\" (UniqueName: \"kubernetes.io/projected/1a2ca818-6395-47ff-a251-e646566f573f-kube-api-access-x7dn7\") pod \"cert-manager-cainjector-559cdd8c87-622wz\" (UID: \"1a2ca818-6395-47ff-a251-e646566f573f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-622wz" Jul 09 04:37:33.194452 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:33.194430 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1a2ca818-6395-47ff-a251-e646566f573f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-622wz\" (UID: \"1a2ca818-6395-47ff-a251-e646566f573f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-622wz" Jul 09 04:37:33.194586 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:33.194568 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7dn7\" (UniqueName: \"kubernetes.io/projected/1a2ca818-6395-47ff-a251-e646566f573f-kube-api-access-x7dn7\") pod \"cert-manager-cainjector-559cdd8c87-622wz\" (UID: \"1a2ca818-6395-47ff-a251-e646566f573f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-622wz" Jul 09 04:37:33.305457 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:33.305422 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-622wz" Jul 09 04:37:33.431763 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:33.431729 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-622wz"] Jul 09 04:37:33.434887 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:37:33.434866 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1a2ca818_6395_47ff_a251_e646566f573f.slice/crio-118be6291d27594522edc557962ee5cd5af6fdc2e7d27455961a6b5891f7efe0 WatchSource:0}: Error finding container 118be6291d27594522edc557962ee5cd5af6fdc2e7d27455961a6b5891f7efe0: Status 404 returned error can't find the container with id 118be6291d27594522edc557962ee5cd5af6fdc2e7d27455961a6b5891f7efe0 Jul 09 04:37:33.437303 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:33.437256 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 04:37:33.499692 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:33.499656 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-622wz" event={"ID":"1a2ca818-6395-47ff-a251-e646566f573f","Type":"ContainerStarted","Data":"118be6291d27594522edc557962ee5cd5af6fdc2e7d27455961a6b5891f7efe0"} Jul 09 04:37:35.076129 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:35.075897 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:37:35.128851 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:35.128719 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:37:36.512727 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:36.512694 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-622wz" event={"ID":"1a2ca818-6395-47ff-a251-e646566f573f","Type":"ContainerStarted","Data":"00028399f8c144e67b404b56f3178c118087b938421ac96338c653856de9e75d"} Jul 09 04:37:36.527791 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:36.527736 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-622wz" podStartSLOduration=1.565218621 podStartE2EDuration="4.527718173s" totalImagesPullingTime="2.962499552s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:37:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:37:36.526926424 +0000 UTC m=+328.778450003" watchObservedRunningTime="2026-07-09 04:37:36.527718173 +0000 UTC m=+328.779241751" Jul 09 04:37:40.531671 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:40.531639 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:37:46.491242 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:46.491208 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:37:46.586534 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:46.586501 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:37:52.684430 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:52.684396 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:37:52.784256 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:52.784217 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:37:54.712488 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:54.712454 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:37:59.445665 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:59.445620 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:37:59.546744 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:37:59.546714 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:04.964372 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:04.964306 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:05.108084 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:05.108042 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:07.282822 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:07.282772 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:07.393487 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:07.393438 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:07.900987 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:07.900958 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:08.142503 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:08.142473 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:08.251073 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:08.251042 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:08.316520 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:08.316478 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-94ccb68c4-j2rhb"] Jul 09 04:38:08.319705 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:08.319681 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-94ccb68c4-j2rhb" Jul 09 04:38:08.324634 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:08.324606 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 09 04:38:08.324772 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:08.324612 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 09 04:38:08.324772 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:08.324668 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 09 04:38:08.324924 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:08.324784 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-w4cln\"" Jul 09 04:38:08.324987 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:08.324939 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 09 04:38:08.344693 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:08.344664 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-94ccb68c4-j2rhb"] Jul 09 04:38:08.357433 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:08.357403 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8857ee0d-be11-43d5-9732-8418776e2f22-tmp\") pod \"opendatahub-operator-controller-manager-94ccb68c4-j2rhb\" (UID: \"8857ee0d-be11-43d5-9732-8418776e2f22\") " pod="opendatahub/opendatahub-operator-controller-manager-94ccb68c4-j2rhb" Jul 09 04:38:08.357591 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:08.357437 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8857ee0d-be11-43d5-9732-8418776e2f22-webhook-cert\") pod \"opendatahub-operator-controller-manager-94ccb68c4-j2rhb\" (UID: \"8857ee0d-be11-43d5-9732-8418776e2f22\") " pod="opendatahub/opendatahub-operator-controller-manager-94ccb68c4-j2rhb" Jul 09 04:38:08.357591 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:08.357464 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qz2fb\" (UniqueName: \"kubernetes.io/projected/8857ee0d-be11-43d5-9732-8418776e2f22-kube-api-access-qz2fb\") pod \"opendatahub-operator-controller-manager-94ccb68c4-j2rhb\" (UID: \"8857ee0d-be11-43d5-9732-8418776e2f22\") " pod="opendatahub/opendatahub-operator-controller-manager-94ccb68c4-j2rhb" Jul 09 04:38:08.357591 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:08.357566 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8857ee0d-be11-43d5-9732-8418776e2f22-manifests\") pod \"opendatahub-operator-controller-manager-94ccb68c4-j2rhb\" (UID: \"8857ee0d-be11-43d5-9732-8418776e2f22\") " pod="opendatahub/opendatahub-operator-controller-manager-94ccb68c4-j2rhb" Jul 09 04:38:08.357724 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:08.357602 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8857ee0d-be11-43d5-9732-8418776e2f22-apiservice-cert\") pod \"opendatahub-operator-controller-manager-94ccb68c4-j2rhb\" (UID: \"8857ee0d-be11-43d5-9732-8418776e2f22\") " pod="opendatahub/opendatahub-operator-controller-manager-94ccb68c4-j2rhb" Jul 09 04:38:08.458712 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:08.458671 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8857ee0d-be11-43d5-9732-8418776e2f22-tmp\") pod \"opendatahub-operator-controller-manager-94ccb68c4-j2rhb\" (UID: \"8857ee0d-be11-43d5-9732-8418776e2f22\") " pod="opendatahub/opendatahub-operator-controller-manager-94ccb68c4-j2rhb" Jul 09 04:38:08.458915 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:08.458722 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8857ee0d-be11-43d5-9732-8418776e2f22-webhook-cert\") pod \"opendatahub-operator-controller-manager-94ccb68c4-j2rhb\" (UID: \"8857ee0d-be11-43d5-9732-8418776e2f22\") " pod="opendatahub/opendatahub-operator-controller-manager-94ccb68c4-j2rhb" Jul 09 04:38:08.458915 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:08.458843 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qz2fb\" (UniqueName: \"kubernetes.io/projected/8857ee0d-be11-43d5-9732-8418776e2f22-kube-api-access-qz2fb\") pod \"opendatahub-operator-controller-manager-94ccb68c4-j2rhb\" (UID: \"8857ee0d-be11-43d5-9732-8418776e2f22\") " pod="opendatahub/opendatahub-operator-controller-manager-94ccb68c4-j2rhb" Jul 09 04:38:08.458915 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:08.458907 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8857ee0d-be11-43d5-9732-8418776e2f22-manifests\") pod \"opendatahub-operator-controller-manager-94ccb68c4-j2rhb\" (UID: \"8857ee0d-be11-43d5-9732-8418776e2f22\") " pod="opendatahub/opendatahub-operator-controller-manager-94ccb68c4-j2rhb" Jul 09 04:38:08.459096 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:08.458976 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8857ee0d-be11-43d5-9732-8418776e2f22-apiservice-cert\") pod \"opendatahub-operator-controller-manager-94ccb68c4-j2rhb\" (UID: \"8857ee0d-be11-43d5-9732-8418776e2f22\") " pod="opendatahub/opendatahub-operator-controller-manager-94ccb68c4-j2rhb" Jul 09 04:38:08.459259 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:08.459229 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8857ee0d-be11-43d5-9732-8418776e2f22-manifests\") pod \"opendatahub-operator-controller-manager-94ccb68c4-j2rhb\" (UID: \"8857ee0d-be11-43d5-9732-8418776e2f22\") " pod="opendatahub/opendatahub-operator-controller-manager-94ccb68c4-j2rhb" Jul 09 04:38:08.461109 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:08.461088 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8857ee0d-be11-43d5-9732-8418776e2f22-tmp\") pod \"opendatahub-operator-controller-manager-94ccb68c4-j2rhb\" (UID: \"8857ee0d-be11-43d5-9732-8418776e2f22\") " pod="opendatahub/opendatahub-operator-controller-manager-94ccb68c4-j2rhb" Jul 09 04:38:08.461220 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:08.461149 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8857ee0d-be11-43d5-9732-8418776e2f22-webhook-cert\") pod \"opendatahub-operator-controller-manager-94ccb68c4-j2rhb\" (UID: \"8857ee0d-be11-43d5-9732-8418776e2f22\") " pod="opendatahub/opendatahub-operator-controller-manager-94ccb68c4-j2rhb" Jul 09 04:38:08.461273 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:08.461216 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8857ee0d-be11-43d5-9732-8418776e2f22-apiservice-cert\") pod \"opendatahub-operator-controller-manager-94ccb68c4-j2rhb\" (UID: \"8857ee0d-be11-43d5-9732-8418776e2f22\") " pod="opendatahub/opendatahub-operator-controller-manager-94ccb68c4-j2rhb" Jul 09 04:38:08.473095 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:08.473067 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qz2fb\" (UniqueName: \"kubernetes.io/projected/8857ee0d-be11-43d5-9732-8418776e2f22-kube-api-access-qz2fb\") pod \"opendatahub-operator-controller-manager-94ccb68c4-j2rhb\" (UID: \"8857ee0d-be11-43d5-9732-8418776e2f22\") " pod="opendatahub/opendatahub-operator-controller-manager-94ccb68c4-j2rhb" Jul 09 04:38:08.530819 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:08.530741 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:08.629526 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:08.629491 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-94ccb68c4-j2rhb" Jul 09 04:38:08.630998 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:08.630971 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:08.731059 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:08.731020 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:08.763460 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:08.763433 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-94ccb68c4-j2rhb"] Jul 09 04:38:08.765591 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:38:08.765562 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8857ee0d_be11_43d5_9732_8418776e2f22.slice/crio-9b58d143adfaa29469fa95318cb2e763ed1b487d52e07f0eac826298f3ad11d3 WatchSource:0}: Error finding container 9b58d143adfaa29469fa95318cb2e763ed1b487d52e07f0eac826298f3ad11d3: Status 404 returned error can't find the container with id 9b58d143adfaa29469fa95318cb2e763ed1b487d52e07f0eac826298f3ad11d3 Jul 09 04:38:08.835561 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:08.835442 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:08.903511 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:08.903476 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:09.013468 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:09.013436 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:09.626687 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:09.626651 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-94ccb68c4-j2rhb" event={"ID":"8857ee0d-be11-43d5-9732-8418776e2f22","Type":"ContainerStarted","Data":"9b58d143adfaa29469fa95318cb2e763ed1b487d52e07f0eac826298f3ad11d3"} Jul 09 04:38:12.640587 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:12.640545 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-94ccb68c4-j2rhb" event={"ID":"8857ee0d-be11-43d5-9732-8418776e2f22","Type":"ContainerStarted","Data":"e13a29362978887b04931e512692204d9dfb55a48e23a78b98aa9c97555b0f96"} Jul 09 04:38:13.649896 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:13.649799 2565 generic.go:356] "Generic (PLEG): container finished" podID="8857ee0d-be11-43d5-9732-8418776e2f22" containerID="e13a29362978887b04931e512692204d9dfb55a48e23a78b98aa9c97555b0f96" exitCode=0 Jul 09 04:38:13.649896 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:13.649881 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-94ccb68c4-j2rhb" event={"ID":"8857ee0d-be11-43d5-9732-8418776e2f22","Type":"ContainerDied","Data":"e13a29362978887b04931e512692204d9dfb55a48e23a78b98aa9c97555b0f96"} Jul 09 04:38:14.655074 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:14.655046 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-94ccb68c4-j2rhb" event={"ID":"8857ee0d-be11-43d5-9732-8418776e2f22","Type":"ContainerStarted","Data":"63255a3ae8cffd143266132d32f9f59005aa95565414b5b979cfacd664e07ab0"} Jul 09 04:38:14.655447 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:14.655175 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-94ccb68c4-j2rhb" Jul 09 04:38:14.675873 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:14.675824 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-94ccb68c4-j2rhb" podStartSLOduration=1.787520225 podStartE2EDuration="6.675808548s" totalImagesPullingTime="3.84875208s" totalInitContainerRuntime="1.039536243s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:38:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:38:14.675446856 +0000 UTC m=+366.926970434" watchObservedRunningTime="2026-07-09 04:38:14.675808548 +0000 UTC m=+366.927332125" Jul 09 04:38:19.423249 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:19.423209 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:19.520118 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:19.520080 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:25.661617 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:25.661583 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-94ccb68c4-j2rhb" Jul 09 04:38:27.560556 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:27.560520 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:27.718569 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:27.718538 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:28.209889 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:28.209843 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:28.248377 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:28.248330 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7c47bc47d8-7hn5h"] Jul 09 04:38:28.258405 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:28.258375 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c47bc47d8-7hn5h" Jul 09 04:38:28.261427 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:28.261400 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 09 04:38:28.261554 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:28.261448 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 09 04:38:28.262568 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:28.262546 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 09 04:38:28.262627 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:28.262582 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-vct28\"" Jul 09 04:38:28.262627 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:28.262620 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 09 04:38:28.268658 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:28.268635 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c47bc47d8-7hn5h"] Jul 09 04:38:28.322356 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:28.322316 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmj22\" (UniqueName: \"kubernetes.io/projected/90ef9e78-acbb-4003-977b-0e8dfa947e8f-kube-api-access-wmj22\") pod \"kube-auth-proxy-7c47bc47d8-7hn5h\" (UID: \"90ef9e78-acbb-4003-977b-0e8dfa947e8f\") " pod="openshift-ingress/kube-auth-proxy-7c47bc47d8-7hn5h" Jul 09 04:38:28.322531 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:28.322454 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90ef9e78-acbb-4003-977b-0e8dfa947e8f-tls-certs\") pod \"kube-auth-proxy-7c47bc47d8-7hn5h\" (UID: \"90ef9e78-acbb-4003-977b-0e8dfa947e8f\") " pod="openshift-ingress/kube-auth-proxy-7c47bc47d8-7hn5h" Jul 09 04:38:28.322531 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:28.322508 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90ef9e78-acbb-4003-977b-0e8dfa947e8f-tmp\") pod \"kube-auth-proxy-7c47bc47d8-7hn5h\" (UID: \"90ef9e78-acbb-4003-977b-0e8dfa947e8f\") " pod="openshift-ingress/kube-auth-proxy-7c47bc47d8-7hn5h" Jul 09 04:38:28.418032 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:28.417989 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-lzxrb"] Jul 09 04:38:28.420032 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:28.420011 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-lzxrb" Jul 09 04:38:28.423059 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:28.423030 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90ef9e78-acbb-4003-977b-0e8dfa947e8f-tls-certs\") pod \"kube-auth-proxy-7c47bc47d8-7hn5h\" (UID: \"90ef9e78-acbb-4003-977b-0e8dfa947e8f\") " pod="openshift-ingress/kube-auth-proxy-7c47bc47d8-7hn5h" Jul 09 04:38:28.423211 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:28.423085 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90ef9e78-acbb-4003-977b-0e8dfa947e8f-tmp\") pod \"kube-auth-proxy-7c47bc47d8-7hn5h\" (UID: \"90ef9e78-acbb-4003-977b-0e8dfa947e8f\") " pod="openshift-ingress/kube-auth-proxy-7c47bc47d8-7hn5h" Jul 09 04:38:28.423211 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:28.423141 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmj22\" (UniqueName: \"kubernetes.io/projected/90ef9e78-acbb-4003-977b-0e8dfa947e8f-kube-api-access-wmj22\") pod \"kube-auth-proxy-7c47bc47d8-7hn5h\" (UID: \"90ef9e78-acbb-4003-977b-0e8dfa947e8f\") " pod="openshift-ingress/kube-auth-proxy-7c47bc47d8-7hn5h" Jul 09 04:38:28.423853 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:28.423831 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 09 04:38:28.424091 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:28.423962 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-7544z\"" Jul 09 04:38:28.425850 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:28.425819 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90ef9e78-acbb-4003-977b-0e8dfa947e8f-tmp\") pod \"kube-auth-proxy-7c47bc47d8-7hn5h\" (UID: \"90ef9e78-acbb-4003-977b-0e8dfa947e8f\") " pod="openshift-ingress/kube-auth-proxy-7c47bc47d8-7hn5h" Jul 09 04:38:28.426035 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:28.426020 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90ef9e78-acbb-4003-977b-0e8dfa947e8f-tls-certs\") pod \"kube-auth-proxy-7c47bc47d8-7hn5h\" (UID: \"90ef9e78-acbb-4003-977b-0e8dfa947e8f\") " pod="openshift-ingress/kube-auth-proxy-7c47bc47d8-7hn5h" Jul 09 04:38:28.442231 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:28.442199 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:28.443532 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:28.443507 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-lzxrb"] Jul 09 04:38:28.447596 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:28.447573 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmj22\" (UniqueName: \"kubernetes.io/projected/90ef9e78-acbb-4003-977b-0e8dfa947e8f-kube-api-access-wmj22\") pod \"kube-auth-proxy-7c47bc47d8-7hn5h\" (UID: \"90ef9e78-acbb-4003-977b-0e8dfa947e8f\") " pod="openshift-ingress/kube-auth-proxy-7c47bc47d8-7hn5h" Jul 09 04:38:28.527436 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:28.525710 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fb4b96ac-6cd9-4b45-a2b9-89667201bb54-cert\") pod \"odh-model-controller-7998c67dc4-lzxrb\" (UID: \"fb4b96ac-6cd9-4b45-a2b9-89667201bb54\") " pod="opendatahub/odh-model-controller-7998c67dc4-lzxrb" Jul 09 04:38:28.527436 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:28.525801 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njmlh\" (UniqueName: \"kubernetes.io/projected/fb4b96ac-6cd9-4b45-a2b9-89667201bb54-kube-api-access-njmlh\") pod \"odh-model-controller-7998c67dc4-lzxrb\" (UID: \"fb4b96ac-6cd9-4b45-a2b9-89667201bb54\") " pod="opendatahub/odh-model-controller-7998c67dc4-lzxrb" Jul 09 04:38:28.527436 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:28.527363 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:28.567561 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:28.567523 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c47bc47d8-7hn5h" Jul 09 04:38:28.624810 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:28.624770 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:28.626894 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:28.626866 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fb4b96ac-6cd9-4b45-a2b9-89667201bb54-cert\") pod \"odh-model-controller-7998c67dc4-lzxrb\" (UID: \"fb4b96ac-6cd9-4b45-a2b9-89667201bb54\") " pod="opendatahub/odh-model-controller-7998c67dc4-lzxrb" Jul 09 04:38:28.627130 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:28.626979 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njmlh\" (UniqueName: \"kubernetes.io/projected/fb4b96ac-6cd9-4b45-a2b9-89667201bb54-kube-api-access-njmlh\") pod \"odh-model-controller-7998c67dc4-lzxrb\" (UID: \"fb4b96ac-6cd9-4b45-a2b9-89667201bb54\") " pod="opendatahub/odh-model-controller-7998c67dc4-lzxrb" Jul 09 04:38:28.627130 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:38:28.627098 2565 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 09 04:38:28.627458 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:38:28.627171 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb4b96ac-6cd9-4b45-a2b9-89667201bb54-cert podName:fb4b96ac-6cd9-4b45-a2b9-89667201bb54 nodeName:}" failed. No retries permitted until 2026-07-09 04:38:29.127147016 +0000 UTC m=+381.378670585 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/fb4b96ac-6cd9-4b45-a2b9-89667201bb54-cert") pod "odh-model-controller-7998c67dc4-lzxrb" (UID: "fb4b96ac-6cd9-4b45-a2b9-89667201bb54") : secret "odh-model-controller-webhook-cert" not found Jul 09 04:38:28.637195 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:28.637169 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njmlh\" (UniqueName: \"kubernetes.io/projected/fb4b96ac-6cd9-4b45-a2b9-89667201bb54-kube-api-access-njmlh\") pod \"odh-model-controller-7998c67dc4-lzxrb\" (UID: \"fb4b96ac-6cd9-4b45-a2b9-89667201bb54\") " pod="opendatahub/odh-model-controller-7998c67dc4-lzxrb" Jul 09 04:38:28.693589 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:28.693565 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c47bc47d8-7hn5h"] Jul 09 04:38:28.696215 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:38:28.696184 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90ef9e78_acbb_4003_977b_0e8dfa947e8f.slice/crio-6c3203f702bbfe66cf918e43be5fb12c980048ef6c6102c89b6312441df684d6 WatchSource:0}: Error finding container 6c3203f702bbfe66cf918e43be5fb12c980048ef6c6102c89b6312441df684d6: Status 404 returned error can't find the container with id 6c3203f702bbfe66cf918e43be5fb12c980048ef6c6102c89b6312441df684d6 Jul 09 04:38:28.705661 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:28.705637 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c47bc47d8-7hn5h" event={"ID":"90ef9e78-acbb-4003-977b-0e8dfa947e8f","Type":"ContainerStarted","Data":"6c3203f702bbfe66cf918e43be5fb12c980048ef6c6102c89b6312441df684d6"} Jul 09 04:38:29.017416 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.017385 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:29.130833 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.130802 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fb4b96ac-6cd9-4b45-a2b9-89667201bb54-cert\") pod \"odh-model-controller-7998c67dc4-lzxrb\" (UID: \"fb4b96ac-6cd9-4b45-a2b9-89667201bb54\") " pod="opendatahub/odh-model-controller-7998c67dc4-lzxrb" Jul 09 04:38:29.133347 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.133311 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fb4b96ac-6cd9-4b45-a2b9-89667201bb54-cert\") pod \"odh-model-controller-7998c67dc4-lzxrb\" (UID: \"fb4b96ac-6cd9-4b45-a2b9-89667201bb54\") " pod="opendatahub/odh-model-controller-7998c67dc4-lzxrb" Jul 09 04:38:29.200161 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.200128 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9rgfm"] Jul 09 04:38:29.203028 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.203000 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-9rgfm" Jul 09 04:38:29.205921 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.205903 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 09 04:38:29.220398 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.220323 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9rgfm"] Jul 09 04:38:29.226542 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.226499 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:29.332504 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.332423 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91e3e24b-5366-4fea-a415-cd9e2bb46439-tls-certs\") pod \"model-serving-api-54d89bd5fc-9rgfm\" (UID: \"91e3e24b-5366-4fea-a415-cd9e2bb46439\") " pod="opendatahub/model-serving-api-54d89bd5fc-9rgfm" Jul 09 04:38:29.332504 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.332478 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-966n9\" (UniqueName: \"kubernetes.io/projected/91e3e24b-5366-4fea-a415-cd9e2bb46439-kube-api-access-966n9\") pod \"model-serving-api-54d89bd5fc-9rgfm\" (UID: \"91e3e24b-5366-4fea-a415-cd9e2bb46439\") " pod="opendatahub/model-serving-api-54d89bd5fc-9rgfm" Jul 09 04:38:29.339388 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.339364 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-lzxrb" Jul 09 04:38:29.434750 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.433915 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91e3e24b-5366-4fea-a415-cd9e2bb46439-tls-certs\") pod \"model-serving-api-54d89bd5fc-9rgfm\" (UID: \"91e3e24b-5366-4fea-a415-cd9e2bb46439\") " pod="opendatahub/model-serving-api-54d89bd5fc-9rgfm" Jul 09 04:38:29.434750 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.434007 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-966n9\" (UniqueName: \"kubernetes.io/projected/91e3e24b-5366-4fea-a415-cd9e2bb46439-kube-api-access-966n9\") pod \"model-serving-api-54d89bd5fc-9rgfm\" (UID: \"91e3e24b-5366-4fea-a415-cd9e2bb46439\") " pod="opendatahub/model-serving-api-54d89bd5fc-9rgfm" Jul 09 04:38:29.434750 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:38:29.434502 2565 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 09 04:38:29.434750 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:38:29.434571 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91e3e24b-5366-4fea-a415-cd9e2bb46439-tls-certs podName:91e3e24b-5366-4fea-a415-cd9e2bb46439 nodeName:}" failed. No retries permitted until 2026-07-09 04:38:29.934547292 +0000 UTC m=+382.186070858 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/91e3e24b-5366-4fea-a415-cd9e2bb46439-tls-certs") pod "model-serving-api-54d89bd5fc-9rgfm" (UID: "91e3e24b-5366-4fea-a415-cd9e2bb46439") : secret "model-serving-api-tls" not found Jul 09 04:38:29.443150 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.443124 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-966n9\" (UniqueName: \"kubernetes.io/projected/91e3e24b-5366-4fea-a415-cd9e2bb46439-kube-api-access-966n9\") pod \"model-serving-api-54d89bd5fc-9rgfm\" (UID: \"91e3e24b-5366-4fea-a415-cd9e2bb46439\") " pod="opendatahub/model-serving-api-54d89bd5fc-9rgfm" Jul 09 04:38:29.490961 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.490913 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-lzxrb"] Jul 09 04:38:29.493251 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.493227 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:29.495106 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:38:29.495076 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb4b96ac_6cd9_4b45_a2b9_89667201bb54.slice/crio-eeef584378928c28c1b88a0f256cea60603a192a71631c46472ecf95df8dc6ae WatchSource:0}: Error finding container eeef584378928c28c1b88a0f256cea60603a192a71631c46472ecf95df8dc6ae: Status 404 returned error can't find the container with id eeef584378928c28c1b88a0f256cea60603a192a71631c46472ecf95df8dc6ae Jul 09 04:38:29.503209 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.503188 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb"] Jul 09 04:38:29.505759 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.505744 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb" Jul 09 04:38:29.507831 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.507812 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 09 04:38:29.507927 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.507866 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 09 04:38:29.507927 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.507813 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 09 04:38:29.508140 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.508121 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 09 04:38:29.508194 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.508176 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-hd9n5\"" Jul 09 04:38:29.523925 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.523897 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb"] Jul 09 04:38:29.586619 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.586511 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:29.635171 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.635132 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-qmvpb\" (UID: \"ed37cd22-d4f2-4382-a61d-e18158b1e5c7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb" Jul 09 04:38:29.635358 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.635178 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-cacerts\") pod \"istiod-openshift-gateway-7574459656-qmvpb\" (UID: \"ed37cd22-d4f2-4382-a61d-e18158b1e5c7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb" Jul 09 04:38:29.635358 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.635269 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-qmvpb\" (UID: \"ed37cd22-d4f2-4382-a61d-e18158b1e5c7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb" Jul 09 04:38:29.635358 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.635287 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nms68\" (UniqueName: \"kubernetes.io/projected/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-kube-api-access-nms68\") pod \"istiod-openshift-gateway-7574459656-qmvpb\" (UID: \"ed37cd22-d4f2-4382-a61d-e18158b1e5c7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb" Jul 09 04:38:29.635358 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.635327 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-local-certs\") pod \"istiod-openshift-gateway-7574459656-qmvpb\" (UID: \"ed37cd22-d4f2-4382-a61d-e18158b1e5c7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb" Jul 09 04:38:29.635548 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.635376 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-istio-token\") pod \"istiod-openshift-gateway-7574459656-qmvpb\" (UID: \"ed37cd22-d4f2-4382-a61d-e18158b1e5c7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb" Jul 09 04:38:29.635548 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.635407 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-qmvpb\" (UID: \"ed37cd22-d4f2-4382-a61d-e18158b1e5c7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb" Jul 09 04:38:29.710425 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.710393 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-lzxrb" event={"ID":"fb4b96ac-6cd9-4b45-a2b9-89667201bb54","Type":"ContainerStarted","Data":"eeef584378928c28c1b88a0f256cea60603a192a71631c46472ecf95df8dc6ae"} Jul 09 04:38:29.735781 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.735738 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-qmvpb\" (UID: \"ed37cd22-d4f2-4382-a61d-e18158b1e5c7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb" Jul 09 04:38:29.735781 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.735789 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-cacerts\") pod \"istiod-openshift-gateway-7574459656-qmvpb\" (UID: \"ed37cd22-d4f2-4382-a61d-e18158b1e5c7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb" Jul 09 04:38:29.735911 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.735834 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-qmvpb\" (UID: \"ed37cd22-d4f2-4382-a61d-e18158b1e5c7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb" Jul 09 04:38:29.735911 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.735851 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nms68\" (UniqueName: \"kubernetes.io/projected/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-kube-api-access-nms68\") pod \"istiod-openshift-gateway-7574459656-qmvpb\" (UID: \"ed37cd22-d4f2-4382-a61d-e18158b1e5c7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb" Jul 09 04:38:29.735911 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.735877 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-local-certs\") pod \"istiod-openshift-gateway-7574459656-qmvpb\" (UID: \"ed37cd22-d4f2-4382-a61d-e18158b1e5c7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb" Jul 09 04:38:29.735911 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.735901 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-istio-token\") pod \"istiod-openshift-gateway-7574459656-qmvpb\" (UID: \"ed37cd22-d4f2-4382-a61d-e18158b1e5c7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb" Jul 09 04:38:29.736090 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.735927 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-qmvpb\" (UID: \"ed37cd22-d4f2-4382-a61d-e18158b1e5c7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb" Jul 09 04:38:29.736480 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.736440 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-qmvpb\" (UID: \"ed37cd22-d4f2-4382-a61d-e18158b1e5c7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb" Jul 09 04:38:29.738402 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.738372 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-qmvpb\" (UID: \"ed37cd22-d4f2-4382-a61d-e18158b1e5c7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb" Jul 09 04:38:29.738520 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.738405 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-qmvpb\" (UID: \"ed37cd22-d4f2-4382-a61d-e18158b1e5c7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb" Jul 09 04:38:29.738520 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.738372 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-local-certs\") pod \"istiod-openshift-gateway-7574459656-qmvpb\" (UID: \"ed37cd22-d4f2-4382-a61d-e18158b1e5c7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb" Jul 09 04:38:29.738602 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.738512 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-cacerts\") pod \"istiod-openshift-gateway-7574459656-qmvpb\" (UID: \"ed37cd22-d4f2-4382-a61d-e18158b1e5c7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb" Jul 09 04:38:29.743239 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.743218 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-istio-token\") pod \"istiod-openshift-gateway-7574459656-qmvpb\" (UID: \"ed37cd22-d4f2-4382-a61d-e18158b1e5c7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb" Jul 09 04:38:29.743314 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.743223 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nms68\" (UniqueName: \"kubernetes.io/projected/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-kube-api-access-nms68\") pod \"istiod-openshift-gateway-7574459656-qmvpb\" (UID: \"ed37cd22-d4f2-4382-a61d-e18158b1e5c7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb" Jul 09 04:38:29.815626 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.815591 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb" Jul 09 04:38:29.845299 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.845226 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:29.937958 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.937926 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91e3e24b-5366-4fea-a415-cd9e2bb46439-tls-certs\") pod \"model-serving-api-54d89bd5fc-9rgfm\" (UID: \"91e3e24b-5366-4fea-a415-cd9e2bb46439\") " pod="opendatahub/model-serving-api-54d89bd5fc-9rgfm" Jul 09 04:38:29.940180 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.940154 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91e3e24b-5366-4fea-a415-cd9e2bb46439-tls-certs\") pod \"model-serving-api-54d89bd5fc-9rgfm\" (UID: \"91e3e24b-5366-4fea-a415-cd9e2bb46439\") " pod="opendatahub/model-serving-api-54d89bd5fc-9rgfm" Jul 09 04:38:29.948494 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:29.948464 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb"] Jul 09 04:38:29.950845 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:38:29.950819 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded37cd22_d4f2_4382_a61d_e18158b1e5c7.slice/crio-8d8ea93f70e5c276c388f214dbaf530203220557ac578f8af5b247bbf3e1abc2 WatchSource:0}: Error finding container 8d8ea93f70e5c276c388f214dbaf530203220557ac578f8af5b247bbf3e1abc2: Status 404 returned error can't find the container with id 8d8ea93f70e5c276c388f214dbaf530203220557ac578f8af5b247bbf3e1abc2 Jul 09 04:38:30.115441 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:30.115355 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-9rgfm" Jul 09 04:38:30.259940 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:30.259903 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9rgfm"] Jul 09 04:38:30.263475 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:38:30.263441 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91e3e24b_5366_4fea_a415_cd9e2bb46439.slice/crio-6766ad70954b8b57eb8fc0fc8e13db5a8888ee61fd1cda4997a49e28708977d2 WatchSource:0}: Error finding container 6766ad70954b8b57eb8fc0fc8e13db5a8888ee61fd1cda4997a49e28708977d2: Status 404 returned error can't find the container with id 6766ad70954b8b57eb8fc0fc8e13db5a8888ee61fd1cda4997a49e28708977d2 Jul 09 04:38:30.374723 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:30.374634 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:30.719224 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:30.719187 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-9rgfm" event={"ID":"91e3e24b-5366-4fea-a415-cd9e2bb46439","Type":"ContainerStarted","Data":"6766ad70954b8b57eb8fc0fc8e13db5a8888ee61fd1cda4997a49e28708977d2"} Jul 09 04:38:30.721258 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:30.721211 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb" event={"ID":"ed37cd22-d4f2-4382-a61d-e18158b1e5c7","Type":"ContainerStarted","Data":"8d8ea93f70e5c276c388f214dbaf530203220557ac578f8af5b247bbf3e1abc2"} Jul 09 04:38:34.550159 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:34.550126 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:38:34.550582 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:34.550192 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:38:34.741410 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:34.741376 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c47bc47d8-7hn5h" event={"ID":"90ef9e78-acbb-4003-977b-0e8dfa947e8f","Type":"ContainerStarted","Data":"a11e6dc13accf7f808e8cbe4ba0703bb0f055229c1c27904c54411ba626ad7cc"} Jul 09 04:38:34.743103 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:34.743076 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb" event={"ID":"ed37cd22-d4f2-4382-a61d-e18158b1e5c7","Type":"ContainerStarted","Data":"fdc1515d8925048ccec564fc50ad7d75a10b93fded6cc0d8cdaea754110c0676"} Jul 09 04:38:34.743242 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:34.743141 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb" Jul 09 04:38:34.744638 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:34.744612 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-lzxrb" event={"ID":"fb4b96ac-6cd9-4b45-a2b9-89667201bb54","Type":"ContainerStarted","Data":"ac958ab82537c14f8bdd2aeac38a24a72a3fc563e9c63eefc75a32c32189ca10"} Jul 09 04:38:34.744759 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:34.744686 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-lzxrb" Jul 09 04:38:34.746580 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:34.746552 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-9rgfm" event={"ID":"91e3e24b-5366-4fea-a415-cd9e2bb46439","Type":"ContainerStarted","Data":"3ef3434cb8554e7284d06f8722f37cc4f03a1adc332f689cd76fe0cae12f17d0"} Jul 09 04:38:34.746908 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:34.746890 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-9rgfm" Jul 09 04:38:34.757674 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:34.757633 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7c47bc47d8-7hn5h" podStartSLOduration=0.911567476 podStartE2EDuration="6.75761863s" totalImagesPullingTime="5.846051154s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:38:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:38:34.75628125 +0000 UTC m=+387.007804832" watchObservedRunningTime="2026-07-09 04:38:34.75761863 +0000 UTC m=+387.009142221" Jul 09 04:38:34.773421 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:34.773364 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-9rgfm" podStartSLOduration=1.494232791 podStartE2EDuration="5.773325416s" totalImagesPullingTime="4.279092625s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:38:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:38:34.771226278 +0000 UTC m=+387.022749858" watchObservedRunningTime="2026-07-09 04:38:34.773325416 +0000 UTC m=+387.024848995" Jul 09 04:38:34.786178 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:34.786132 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-lzxrb" podStartSLOduration=1.689745252 podStartE2EDuration="6.786115261s" totalImagesPullingTime="5.096370009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:38:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:38:34.785453195 +0000 UTC m=+387.036976774" watchObservedRunningTime="2026-07-09 04:38:34.786115261 +0000 UTC m=+387.037638840" Jul 09 04:38:34.804295 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:34.804206 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb" podStartSLOduration=1.2070338170000001 podStartE2EDuration="5.804190209s" totalImagesPullingTime="4.597156392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:38:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:38:34.802493113 +0000 UTC m=+387.054016692" watchObservedRunningTime="2026-07-09 04:38:34.804190209 +0000 UTC m=+387.055713787" Jul 09 04:38:35.407914 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:35.407882 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:35.513484 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:35.513451 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:35.751978 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:35.751902 2565 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-qmvpb container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 09 04:38:35.751978 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:35.751960 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb" podUID="ed37cd22-d4f2-4382-a61d-e18158b1e5c7" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 04:38:36.145801 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:36.145709 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:36.373883 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:36.373851 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:36.440475 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:36.440446 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:38.692550 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:38.692514 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:38.751219 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:38.751192 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb" Jul 09 04:38:38.763495 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:38.763461 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-mx8xg"] Jul 09 04:38:38.767103 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:38.767081 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mx8xg" Jul 09 04:38:38.769709 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:38.769687 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-qnk26\"" Jul 09 04:38:38.769709 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:38.769701 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 09 04:38:38.778233 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:38.778174 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-x8wrk"] Jul 09 04:38:38.781563 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:38.781541 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-mx8xg"] Jul 09 04:38:38.781680 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:38.781654 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x8wrk" Jul 09 04:38:38.785121 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:38.785102 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 09 04:38:38.786029 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:38.786008 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-w88w6\"" Jul 09 04:38:38.796449 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:38.796423 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-x8wrk"] Jul 09 04:38:38.797960 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:38.797920 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:38.918830 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:38.918792 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgpzh\" (UniqueName: \"kubernetes.io/projected/bed406fb-4a82-40d6-b190-5cf5033f6867-kube-api-access-hgpzh\") pod \"llmisvc-controller-manager-777c6bd7d5-x8wrk\" (UID: \"bed406fb-4a82-40d6-b190-5cf5033f6867\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x8wrk" Jul 09 04:38:38.919009 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:38.918850 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bed406fb-4a82-40d6-b190-5cf5033f6867-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-x8wrk\" (UID: \"bed406fb-4a82-40d6-b190-5cf5033f6867\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x8wrk" Jul 09 04:38:38.919009 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:38.918968 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/02072840-9f3b-43a1-a940-ac8aa8b1c188-cert\") pod \"kserve-controller-manager-d6f4b65b8-mx8xg\" (UID: \"02072840-9f3b-43a1-a940-ac8aa8b1c188\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mx8xg" Jul 09 04:38:38.919125 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:38.919028 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jzb7\" (UniqueName: \"kubernetes.io/projected/02072840-9f3b-43a1-a940-ac8aa8b1c188-kube-api-access-2jzb7\") pod \"kserve-controller-manager-d6f4b65b8-mx8xg\" (UID: \"02072840-9f3b-43a1-a940-ac8aa8b1c188\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mx8xg" Jul 09 04:38:38.990323 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:38.990240 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:39.019549 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:39.019518 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/02072840-9f3b-43a1-a940-ac8aa8b1c188-cert\") pod \"kserve-controller-manager-d6f4b65b8-mx8xg\" (UID: \"02072840-9f3b-43a1-a940-ac8aa8b1c188\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mx8xg" Jul 09 04:38:39.019740 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:39.019563 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jzb7\" (UniqueName: \"kubernetes.io/projected/02072840-9f3b-43a1-a940-ac8aa8b1c188-kube-api-access-2jzb7\") pod \"kserve-controller-manager-d6f4b65b8-mx8xg\" (UID: \"02072840-9f3b-43a1-a940-ac8aa8b1c188\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mx8xg" Jul 09 04:38:39.019740 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:39.019617 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgpzh\" (UniqueName: \"kubernetes.io/projected/bed406fb-4a82-40d6-b190-5cf5033f6867-kube-api-access-hgpzh\") pod \"llmisvc-controller-manager-777c6bd7d5-x8wrk\" (UID: \"bed406fb-4a82-40d6-b190-5cf5033f6867\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x8wrk" Jul 09 04:38:39.019740 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:38:39.019635 2565 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 09 04:38:39.019740 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:39.019657 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bed406fb-4a82-40d6-b190-5cf5033f6867-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-x8wrk\" (UID: \"bed406fb-4a82-40d6-b190-5cf5033f6867\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x8wrk" Jul 09 04:38:39.019740 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:38:39.019712 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02072840-9f3b-43a1-a940-ac8aa8b1c188-cert podName:02072840-9f3b-43a1-a940-ac8aa8b1c188 nodeName:}" failed. No retries permitted until 2026-07-09 04:38:39.519687526 +0000 UTC m=+391.771211108 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/02072840-9f3b-43a1-a940-ac8aa8b1c188-cert") pod "kserve-controller-manager-d6f4b65b8-mx8xg" (UID: "02072840-9f3b-43a1-a940-ac8aa8b1c188") : secret "kserve-webhook-server-cert" not found Jul 09 04:38:39.022259 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:39.022233 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bed406fb-4a82-40d6-b190-5cf5033f6867-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-x8wrk\" (UID: \"bed406fb-4a82-40d6-b190-5cf5033f6867\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x8wrk" Jul 09 04:38:39.039640 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:39.039607 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jzb7\" (UniqueName: \"kubernetes.io/projected/02072840-9f3b-43a1-a940-ac8aa8b1c188-kube-api-access-2jzb7\") pod \"kserve-controller-manager-d6f4b65b8-mx8xg\" (UID: \"02072840-9f3b-43a1-a940-ac8aa8b1c188\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mx8xg" Jul 09 04:38:39.039761 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:39.039651 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgpzh\" (UniqueName: \"kubernetes.io/projected/bed406fb-4a82-40d6-b190-5cf5033f6867-kube-api-access-hgpzh\") pod \"llmisvc-controller-manager-777c6bd7d5-x8wrk\" (UID: \"bed406fb-4a82-40d6-b190-5cf5033f6867\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x8wrk" Jul 09 04:38:39.091860 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:39.091815 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x8wrk" Jul 09 04:38:39.234611 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:39.234583 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-x8wrk"] Jul 09 04:38:39.236992 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:38:39.236967 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podbed406fb_4a82_40d6_b190_5cf5033f6867.slice/crio-107cdf940c93e1e508e38b8ef703fa2a34731e9ea88b8f7f03001089c206ae51 WatchSource:0}: Error finding container 107cdf940c93e1e508e38b8ef703fa2a34731e9ea88b8f7f03001089c206ae51: Status 404 returned error can't find the container with id 107cdf940c93e1e508e38b8ef703fa2a34731e9ea88b8f7f03001089c206ae51 Jul 09 04:38:39.342303 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:39.342215 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:39.524174 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:39.524100 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/02072840-9f3b-43a1-a940-ac8aa8b1c188-cert\") pod \"kserve-controller-manager-d6f4b65b8-mx8xg\" (UID: \"02072840-9f3b-43a1-a940-ac8aa8b1c188\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mx8xg" Jul 09 04:38:39.526604 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:39.526584 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/02072840-9f3b-43a1-a940-ac8aa8b1c188-cert\") pod \"kserve-controller-manager-d6f4b65b8-mx8xg\" (UID: \"02072840-9f3b-43a1-a940-ac8aa8b1c188\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mx8xg" Jul 09 04:38:39.680010 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:39.679975 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mx8xg" Jul 09 04:38:39.772762 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:39.772734 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x8wrk" event={"ID":"bed406fb-4a82-40d6-b190-5cf5033f6867","Type":"ContainerStarted","Data":"107cdf940c93e1e508e38b8ef703fa2a34731e9ea88b8f7f03001089c206ae51"} Jul 09 04:38:39.828917 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:39.828888 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-mx8xg"] Jul 09 04:38:39.872183 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:38:39.872145 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02072840_9f3b_43a1_a940_ac8aa8b1c188.slice/crio-cc7b92a5d4f9b50f5f6292c7057da7dfd483349d8448e26ac432606f36798a80 WatchSource:0}: Error finding container cc7b92a5d4f9b50f5f6292c7057da7dfd483349d8448e26ac432606f36798a80: Status 404 returned error can't find the container with id cc7b92a5d4f9b50f5f6292c7057da7dfd483349d8448e26ac432606f36798a80 Jul 09 04:38:39.982195 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:39.982032 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:40.086513 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:40.086478 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:40.210107 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:40.210078 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:40.247923 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:40.247855 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb"] Jul 09 04:38:40.248055 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:40.248021 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb" podUID="ed37cd22-d4f2-4382-a61d-e18158b1e5c7" containerName="discovery" containerID="cri-o://fdc1515d8925048ccec564fc50ad7d75a10b93fded6cc0d8cdaea754110c0676" gracePeriod=30 Jul 09 04:38:40.621637 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:40.621609 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb" Jul 09 04:38:40.734223 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:40.734191 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-istio-kubeconfig\") pod \"ed37cd22-d4f2-4382-a61d-e18158b1e5c7\" (UID: \"ed37cd22-d4f2-4382-a61d-e18158b1e5c7\") " Jul 09 04:38:40.734223 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:40.734230 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-local-certs\") pod \"ed37cd22-d4f2-4382-a61d-e18158b1e5c7\" (UID: \"ed37cd22-d4f2-4382-a61d-e18158b1e5c7\") " Jul 09 04:38:40.734486 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:40.734257 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-cacerts\") pod \"ed37cd22-d4f2-4382-a61d-e18158b1e5c7\" (UID: \"ed37cd22-d4f2-4382-a61d-e18158b1e5c7\") " Jul 09 04:38:40.734486 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:40.734290 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-istio-csr-ca-configmap\") pod \"ed37cd22-d4f2-4382-a61d-e18158b1e5c7\" (UID: \"ed37cd22-d4f2-4382-a61d-e18158b1e5c7\") " Jul 09 04:38:40.734486 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:40.734430 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-istio-csr-dns-cert\") pod \"ed37cd22-d4f2-4382-a61d-e18158b1e5c7\" (UID: \"ed37cd22-d4f2-4382-a61d-e18158b1e5c7\") " Jul 09 04:38:40.734486 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:40.734480 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nms68\" (UniqueName: \"kubernetes.io/projected/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-kube-api-access-nms68\") pod \"ed37cd22-d4f2-4382-a61d-e18158b1e5c7\" (UID: \"ed37cd22-d4f2-4382-a61d-e18158b1e5c7\") " Jul 09 04:38:40.734687 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:40.734561 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-istio-token\") pod \"ed37cd22-d4f2-4382-a61d-e18158b1e5c7\" (UID: \"ed37cd22-d4f2-4382-a61d-e18158b1e5c7\") " Jul 09 04:38:40.734751 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:40.734716 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "ed37cd22-d4f2-4382-a61d-e18158b1e5c7" (UID: "ed37cd22-d4f2-4382-a61d-e18158b1e5c7"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:38:40.734853 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:40.734835 2565 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-istio-csr-ca-configmap\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:38:40.737371 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:40.737306 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-cacerts" (OuterVolumeSpecName: "cacerts") pod "ed37cd22-d4f2-4382-a61d-e18158b1e5c7" (UID: "ed37cd22-d4f2-4382-a61d-e18158b1e5c7"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:38:40.737498 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:40.737422 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "ed37cd22-d4f2-4382-a61d-e18158b1e5c7" (UID: "ed37cd22-d4f2-4382-a61d-e18158b1e5c7"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:38:40.737498 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:40.737449 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-kube-api-access-nms68" (OuterVolumeSpecName: "kube-api-access-nms68") pod "ed37cd22-d4f2-4382-a61d-e18158b1e5c7" (UID: "ed37cd22-d4f2-4382-a61d-e18158b1e5c7"). InnerVolumeSpecName "kube-api-access-nms68". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:38:40.737610 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:40.737559 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-local-certs" (OuterVolumeSpecName: "local-certs") pod "ed37cd22-d4f2-4382-a61d-e18158b1e5c7" (UID: "ed37cd22-d4f2-4382-a61d-e18158b1e5c7"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:38:40.737805 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:40.737764 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-istio-token" (OuterVolumeSpecName: "istio-token") pod "ed37cd22-d4f2-4382-a61d-e18158b1e5c7" (UID: "ed37cd22-d4f2-4382-a61d-e18158b1e5c7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:38:40.737805 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:40.737787 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "ed37cd22-d4f2-4382-a61d-e18158b1e5c7" (UID: "ed37cd22-d4f2-4382-a61d-e18158b1e5c7"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:38:40.778576 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:40.778484 2565 generic.go:356] "Generic (PLEG): container finished" podID="ed37cd22-d4f2-4382-a61d-e18158b1e5c7" containerID="fdc1515d8925048ccec564fc50ad7d75a10b93fded6cc0d8cdaea754110c0676" exitCode=0 Jul 09 04:38:40.778576 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:40.778534 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb" Jul 09 04:38:40.778576 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:40.778564 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb" event={"ID":"ed37cd22-d4f2-4382-a61d-e18158b1e5c7","Type":"ContainerDied","Data":"fdc1515d8925048ccec564fc50ad7d75a10b93fded6cc0d8cdaea754110c0676"} Jul 09 04:38:40.779110 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:40.778616 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb" event={"ID":"ed37cd22-d4f2-4382-a61d-e18158b1e5c7","Type":"ContainerDied","Data":"8d8ea93f70e5c276c388f214dbaf530203220557ac578f8af5b247bbf3e1abc2"} Jul 09 04:38:40.779110 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:40.778642 2565 scope.go:117] "RemoveContainer" containerID="fdc1515d8925048ccec564fc50ad7d75a10b93fded6cc0d8cdaea754110c0676" Jul 09 04:38:40.779762 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:40.779697 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mx8xg" event={"ID":"02072840-9f3b-43a1-a940-ac8aa8b1c188","Type":"ContainerStarted","Data":"cc7b92a5d4f9b50f5f6292c7057da7dfd483349d8448e26ac432606f36798a80"} Jul 09 04:38:40.786808 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:40.786786 2565 scope.go:117] "RemoveContainer" containerID="fdc1515d8925048ccec564fc50ad7d75a10b93fded6cc0d8cdaea754110c0676" Jul 09 04:38:40.787048 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:38:40.787030 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdc1515d8925048ccec564fc50ad7d75a10b93fded6cc0d8cdaea754110c0676\": container with ID starting with fdc1515d8925048ccec564fc50ad7d75a10b93fded6cc0d8cdaea754110c0676 not found: ID does not exist" containerID="fdc1515d8925048ccec564fc50ad7d75a10b93fded6cc0d8cdaea754110c0676" Jul 09 04:38:40.787117 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:40.787060 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdc1515d8925048ccec564fc50ad7d75a10b93fded6cc0d8cdaea754110c0676"} err="failed to get container status \"fdc1515d8925048ccec564fc50ad7d75a10b93fded6cc0d8cdaea754110c0676\": rpc error: code = NotFound desc = could not find container \"fdc1515d8925048ccec564fc50ad7d75a10b93fded6cc0d8cdaea754110c0676\": container with ID starting with fdc1515d8925048ccec564fc50ad7d75a10b93fded6cc0d8cdaea754110c0676 not found: ID does not exist" Jul 09 04:38:40.799433 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:40.799411 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb"] Jul 09 04:38:40.811409 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:40.811387 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qmvpb"] Jul 09 04:38:40.835730 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:40.835697 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-istio-token\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:38:40.835730 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:40.835728 2565 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-istio-kubeconfig\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:38:40.835916 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:40.835740 2565 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-local-certs\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:38:40.835916 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:40.835753 2565 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-cacerts\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:38:40.835916 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:40.835764 2565 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-istio-csr-dns-cert\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:38:40.835916 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:40.835777 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nms68\" (UniqueName: \"kubernetes.io/projected/ed37cd22-d4f2-4382-a61d-e18158b1e5c7-kube-api-access-nms68\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:38:42.196555 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:42.196517 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed37cd22-d4f2-4382-a61d-e18158b1e5c7" path="/var/lib/kubelet/pods/ed37cd22-d4f2-4382-a61d-e18158b1e5c7/volumes" Jul 09 04:38:43.798454 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:43.798420 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mx8xg" event={"ID":"02072840-9f3b-43a1-a940-ac8aa8b1c188","Type":"ContainerStarted","Data":"3d1bb28cc5c47536e177b5384f1f01c7d4cd0924e54ddf5d49930db9f3843119"} Jul 09 04:38:43.798906 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:43.798508 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mx8xg" Jul 09 04:38:43.799959 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:43.799938 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x8wrk" event={"ID":"bed406fb-4a82-40d6-b190-5cf5033f6867","Type":"ContainerStarted","Data":"e3f0dfbc3a0e66dd4acaa714ce0da9a32ace98cb6d6013300ee49a335b8723e4"} Jul 09 04:38:43.800081 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:43.799982 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x8wrk" Jul 09 04:38:43.815536 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:43.815487 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mx8xg" podStartSLOduration=2.851990186 podStartE2EDuration="5.815472683s" totalImagesPullingTime="2.963482497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:38:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:38:43.814088471 +0000 UTC m=+396.065612050" watchObservedRunningTime="2026-07-09 04:38:43.815472683 +0000 UTC m=+396.066996261" Jul 09 04:38:43.829894 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:43.829846 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x8wrk" podStartSLOduration=2.234054548 podStartE2EDuration="5.829830912s" totalImagesPullingTime="3.595776364s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:38:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:38:43.829450546 +0000 UTC m=+396.080974136" watchObservedRunningTime="2026-07-09 04:38:43.829830912 +0000 UTC m=+396.081354489" Jul 09 04:38:45.754387 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:45.754356 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-9rgfm" Jul 09 04:38:45.791854 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:45.791826 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-lzxrb" Jul 09 04:38:50.225858 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:50.225823 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:38:50.278847 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:38:50.278817 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:39:14.806178 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:14.806150 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x8wrk" Jul 09 04:39:14.809126 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:14.809104 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mx8xg" Jul 09 04:39:28.959704 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:28.959669 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:39:29.059451 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:29.059421 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:39:29.094200 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:29.094165 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5kbvw"] Jul 09 04:39:29.094475 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:29.094462 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed37cd22-d4f2-4382-a61d-e18158b1e5c7" containerName="discovery" Jul 09 04:39:29.094523 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:29.094477 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed37cd22-d4f2-4382-a61d-e18158b1e5c7" containerName="discovery" Jul 09 04:39:29.094554 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:29.094540 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed37cd22-d4f2-4382-a61d-e18158b1e5c7" containerName="discovery" Jul 09 04:39:29.103844 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:29.103817 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5kbvw" Jul 09 04:39:29.106482 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:29.106455 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 09 04:39:29.107095 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:29.107062 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5kbvw"] Jul 09 04:39:29.107661 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:29.107635 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-ksqf4\"" Jul 09 04:39:29.107815 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:29.107791 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 09 04:39:29.141204 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:29.141173 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:39:29.225285 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:29.225190 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6qgg\" (UniqueName: \"kubernetes.io/projected/4203b6c9-974c-491b-a55d-35d89ebf686c-kube-api-access-b6qgg\") pod \"kuadrant-operator-catalog-5kbvw\" (UID: \"4203b6c9-974c-491b-a55d-35d89ebf686c\") " pod="kuadrant-system/kuadrant-operator-catalog-5kbvw" Jul 09 04:39:29.326250 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:29.326202 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6qgg\" (UniqueName: \"kubernetes.io/projected/4203b6c9-974c-491b-a55d-35d89ebf686c-kube-api-access-b6qgg\") pod \"kuadrant-operator-catalog-5kbvw\" (UID: \"4203b6c9-974c-491b-a55d-35d89ebf686c\") " pod="kuadrant-system/kuadrant-operator-catalog-5kbvw" Jul 09 04:39:29.334563 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:29.334531 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6qgg\" (UniqueName: \"kubernetes.io/projected/4203b6c9-974c-491b-a55d-35d89ebf686c-kube-api-access-b6qgg\") pod \"kuadrant-operator-catalog-5kbvw\" (UID: \"4203b6c9-974c-491b-a55d-35d89ebf686c\") " pod="kuadrant-system/kuadrant-operator-catalog-5kbvw" Jul 09 04:39:29.415064 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:29.415024 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5kbvw" Jul 09 04:39:29.460142 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:29.460104 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5kbvw"] Jul 09 04:39:29.541031 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:29.540952 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5kbvw"] Jul 09 04:39:29.543762 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:39:29.543730 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4203b6c9_974c_491b_a55d_35d89ebf686c.slice/crio-fca849120aaa06f47bc9132293f43e7655c87a087f14909d1bb8bbd2337ca134 WatchSource:0}: Error finding container fca849120aaa06f47bc9132293f43e7655c87a087f14909d1bb8bbd2337ca134: Status 404 returned error can't find the container with id fca849120aaa06f47bc9132293f43e7655c87a087f14909d1bb8bbd2337ca134 Jul 09 04:39:29.670914 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:29.670881 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2mqz2"] Jul 09 04:39:29.673551 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:29.673534 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2mqz2" Jul 09 04:39:29.681834 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:29.681801 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2mqz2"] Jul 09 04:39:29.730226 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:29.730148 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4rz6\" (UniqueName: \"kubernetes.io/projected/3f08431a-d380-4d64-90c5-cea110be4794-kube-api-access-j4rz6\") pod \"kuadrant-operator-catalog-2mqz2\" (UID: \"3f08431a-d380-4d64-90c5-cea110be4794\") " pod="kuadrant-system/kuadrant-operator-catalog-2mqz2" Jul 09 04:39:29.831422 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:29.831312 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4rz6\" (UniqueName: \"kubernetes.io/projected/3f08431a-d380-4d64-90c5-cea110be4794-kube-api-access-j4rz6\") pod \"kuadrant-operator-catalog-2mqz2\" (UID: \"3f08431a-d380-4d64-90c5-cea110be4794\") " pod="kuadrant-system/kuadrant-operator-catalog-2mqz2" Jul 09 04:39:29.838594 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:29.838575 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4rz6\" (UniqueName: \"kubernetes.io/projected/3f08431a-d380-4d64-90c5-cea110be4794-kube-api-access-j4rz6\") pod \"kuadrant-operator-catalog-2mqz2\" (UID: \"3f08431a-d380-4d64-90c5-cea110be4794\") " pod="kuadrant-system/kuadrant-operator-catalog-2mqz2" Jul 09 04:39:29.983544 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:29.983507 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2mqz2" Jul 09 04:39:29.995597 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:29.995561 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5kbvw" event={"ID":"4203b6c9-974c-491b-a55d-35d89ebf686c","Type":"ContainerStarted","Data":"fca849120aaa06f47bc9132293f43e7655c87a087f14909d1bb8bbd2337ca134"} Jul 09 04:39:30.110308 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:30.110264 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2mqz2"] Jul 09 04:39:30.112498 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:39:30.112466 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f08431a_d380_4d64_90c5_cea110be4794.slice/crio-000165ff718615b6e2a95c3476324a713c9ab131783dc0a5188185d98a8ccd82 WatchSource:0}: Error finding container 000165ff718615b6e2a95c3476324a713c9ab131783dc0a5188185d98a8ccd82: Status 404 returned error can't find the container with id 000165ff718615b6e2a95c3476324a713c9ab131783dc0a5188185d98a8ccd82 Jul 09 04:39:31.002599 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:31.002562 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2mqz2" event={"ID":"3f08431a-d380-4d64-90c5-cea110be4794","Type":"ContainerStarted","Data":"000165ff718615b6e2a95c3476324a713c9ab131783dc0a5188185d98a8ccd82"} Jul 09 04:39:32.009400 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:32.009268 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5kbvw" event={"ID":"4203b6c9-974c-491b-a55d-35d89ebf686c","Type":"ContainerStarted","Data":"cabee1b3614f6cd3468b4ca27371141d8ef1dabdb7fa0d30dfc77bc936299e3a"} Jul 09 04:39:32.009400 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:32.009300 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-5kbvw" podUID="4203b6c9-974c-491b-a55d-35d89ebf686c" containerName="registry-server" containerID="cri-o://cabee1b3614f6cd3468b4ca27371141d8ef1dabdb7fa0d30dfc77bc936299e3a" gracePeriod=2 Jul 09 04:39:32.011256 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:32.011226 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2mqz2" event={"ID":"3f08431a-d380-4d64-90c5-cea110be4794","Type":"ContainerStarted","Data":"d6c838b3c69452f87ded953124be267318b00b77db3e0dd6caca2021cf27ceaf"} Jul 09 04:39:32.040359 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:32.036454 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-5kbvw" podStartSLOduration=0.922156018 podStartE2EDuration="3.036436742s" totalImagesPullingTime="2.114280724s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:39:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:39:32.035665268 +0000 UTC m=+444.287188844" watchObservedRunningTime="2026-07-09 04:39:32.036436742 +0000 UTC m=+444.287960320" Jul 09 04:39:32.055526 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:32.055466 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-2mqz2" podStartSLOduration=1.458516224 podStartE2EDuration="3.055446518s" totalImagesPullingTime="1.596930294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:39:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:39:32.055108578 +0000 UTC m=+444.306632157" watchObservedRunningTime="2026-07-09 04:39:32.055446518 +0000 UTC m=+444.306970096" Jul 09 04:39:32.252375 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:32.252333 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-5kbvw_4203b6c9-974c-491b-a55d-35d89ebf686c/registry-server/0.log" Jul 09 04:39:32.254626 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:32.254607 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5kbvw" Jul 09 04:39:32.351616 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:32.351520 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6qgg\" (UniqueName: \"kubernetes.io/projected/4203b6c9-974c-491b-a55d-35d89ebf686c-kube-api-access-b6qgg\") pod \"4203b6c9-974c-491b-a55d-35d89ebf686c\" (UID: \"4203b6c9-974c-491b-a55d-35d89ebf686c\") " Jul 09 04:39:32.353742 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:32.353710 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4203b6c9-974c-491b-a55d-35d89ebf686c-kube-api-access-b6qgg" (OuterVolumeSpecName: "kube-api-access-b6qgg") pod "4203b6c9-974c-491b-a55d-35d89ebf686c" (UID: "4203b6c9-974c-491b-a55d-35d89ebf686c"). InnerVolumeSpecName "kube-api-access-b6qgg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:39:32.452158 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:32.452118 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6qgg\" (UniqueName: \"kubernetes.io/projected/4203b6c9-974c-491b-a55d-35d89ebf686c-kube-api-access-b6qgg\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:39:33.016679 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:33.016604 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-5kbvw_4203b6c9-974c-491b-a55d-35d89ebf686c/registry-server/0.log" Jul 09 04:39:33.016679 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:33.016645 2565 generic.go:356] "Generic (PLEG): container finished" podID="4203b6c9-974c-491b-a55d-35d89ebf686c" containerID="cabee1b3614f6cd3468b4ca27371141d8ef1dabdb7fa0d30dfc77bc936299e3a" exitCode=1 Jul 09 04:39:33.017141 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:33.016692 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5kbvw" Jul 09 04:39:33.017141 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:33.016736 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5kbvw" event={"ID":"4203b6c9-974c-491b-a55d-35d89ebf686c","Type":"ContainerDied","Data":"cabee1b3614f6cd3468b4ca27371141d8ef1dabdb7fa0d30dfc77bc936299e3a"} Jul 09 04:39:33.017141 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:33.016793 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5kbvw" event={"ID":"4203b6c9-974c-491b-a55d-35d89ebf686c","Type":"ContainerDied","Data":"fca849120aaa06f47bc9132293f43e7655c87a087f14909d1bb8bbd2337ca134"} Jul 09 04:39:33.017141 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:33.016825 2565 scope.go:117] "RemoveContainer" containerID="cabee1b3614f6cd3468b4ca27371141d8ef1dabdb7fa0d30dfc77bc936299e3a" Jul 09 04:39:33.025692 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:33.025675 2565 scope.go:117] "RemoveContainer" containerID="cabee1b3614f6cd3468b4ca27371141d8ef1dabdb7fa0d30dfc77bc936299e3a" Jul 09 04:39:33.025939 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:39:33.025917 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cabee1b3614f6cd3468b4ca27371141d8ef1dabdb7fa0d30dfc77bc936299e3a\": container with ID starting with cabee1b3614f6cd3468b4ca27371141d8ef1dabdb7fa0d30dfc77bc936299e3a not found: ID does not exist" containerID="cabee1b3614f6cd3468b4ca27371141d8ef1dabdb7fa0d30dfc77bc936299e3a" Jul 09 04:39:33.025977 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:33.025949 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cabee1b3614f6cd3468b4ca27371141d8ef1dabdb7fa0d30dfc77bc936299e3a"} err="failed to get container status \"cabee1b3614f6cd3468b4ca27371141d8ef1dabdb7fa0d30dfc77bc936299e3a\": rpc error: code = NotFound desc = could not find container \"cabee1b3614f6cd3468b4ca27371141d8ef1dabdb7fa0d30dfc77bc936299e3a\": container with ID starting with cabee1b3614f6cd3468b4ca27371141d8ef1dabdb7fa0d30dfc77bc936299e3a not found: ID does not exist" Jul 09 04:39:33.036242 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:33.036221 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5kbvw"] Jul 09 04:39:33.039047 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:33.039028 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5kbvw"] Jul 09 04:39:34.194488 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:34.194457 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4203b6c9-974c-491b-a55d-35d89ebf686c" path="/var/lib/kubelet/pods/4203b6c9-974c-491b-a55d-35d89ebf686c/volumes" Jul 09 04:39:39.984551 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:39.984499 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-2mqz2" Jul 09 04:39:39.985001 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:39.984649 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-2mqz2" Jul 09 04:39:40.006585 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:40.006555 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-2mqz2" Jul 09 04:39:40.067801 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:40.067775 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-2mqz2" Jul 09 04:39:57.755082 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:57.755050 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:39:58.079188 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:58.079110 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-554bb9dccb-bz2vd"] Jul 09 04:39:58.079435 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:58.079420 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4203b6c9-974c-491b-a55d-35d89ebf686c" containerName="registry-server" Jul 09 04:39:58.079435 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:58.079436 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="4203b6c9-974c-491b-a55d-35d89ebf686c" containerName="registry-server" Jul 09 04:39:58.079538 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:58.079490 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="4203b6c9-974c-491b-a55d-35d89ebf686c" containerName="registry-server" Jul 09 04:39:58.083934 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:58.083912 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-554bb9dccb-bz2vd" Jul 09 04:39:58.093426 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:58.093401 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-554bb9dccb-bz2vd"] Jul 09 04:39:58.138280 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:58.138251 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:39:58.165405 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:58.165374 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d99c210e-193f-416e-9c5e-234f96d978cc-oauth-serving-cert\") pod \"console-554bb9dccb-bz2vd\" (UID: \"d99c210e-193f-416e-9c5e-234f96d978cc\") " pod="openshift-console/console-554bb9dccb-bz2vd" Jul 09 04:39:58.165576 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:58.165413 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d99c210e-193f-416e-9c5e-234f96d978cc-console-serving-cert\") pod \"console-554bb9dccb-bz2vd\" (UID: \"d99c210e-193f-416e-9c5e-234f96d978cc\") " pod="openshift-console/console-554bb9dccb-bz2vd" Jul 09 04:39:58.165576 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:58.165500 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d99c210e-193f-416e-9c5e-234f96d978cc-console-config\") pod \"console-554bb9dccb-bz2vd\" (UID: \"d99c210e-193f-416e-9c5e-234f96d978cc\") " pod="openshift-console/console-554bb9dccb-bz2vd" Jul 09 04:39:58.165576 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:58.165530 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d99c210e-193f-416e-9c5e-234f96d978cc-trusted-ca-bundle\") pod \"console-554bb9dccb-bz2vd\" (UID: \"d99c210e-193f-416e-9c5e-234f96d978cc\") " pod="openshift-console/console-554bb9dccb-bz2vd" Jul 09 04:39:58.165576 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:58.165556 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d99c210e-193f-416e-9c5e-234f96d978cc-console-oauth-config\") pod \"console-554bb9dccb-bz2vd\" (UID: \"d99c210e-193f-416e-9c5e-234f96d978cc\") " pod="openshift-console/console-554bb9dccb-bz2vd" Jul 09 04:39:58.165576 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:58.165572 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8m2w\" (UniqueName: \"kubernetes.io/projected/d99c210e-193f-416e-9c5e-234f96d978cc-kube-api-access-g8m2w\") pod \"console-554bb9dccb-bz2vd\" (UID: \"d99c210e-193f-416e-9c5e-234f96d978cc\") " pod="openshift-console/console-554bb9dccb-bz2vd" Jul 09 04:39:58.165753 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:58.165611 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d99c210e-193f-416e-9c5e-234f96d978cc-service-ca\") pod \"console-554bb9dccb-bz2vd\" (UID: \"d99c210e-193f-416e-9c5e-234f96d978cc\") " pod="openshift-console/console-554bb9dccb-bz2vd" Jul 09 04:39:58.266543 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:58.266510 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d99c210e-193f-416e-9c5e-234f96d978cc-console-config\") pod \"console-554bb9dccb-bz2vd\" (UID: \"d99c210e-193f-416e-9c5e-234f96d978cc\") " pod="openshift-console/console-554bb9dccb-bz2vd" Jul 09 04:39:58.266543 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:58.266541 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d99c210e-193f-416e-9c5e-234f96d978cc-trusted-ca-bundle\") pod \"console-554bb9dccb-bz2vd\" (UID: \"d99c210e-193f-416e-9c5e-234f96d978cc\") " pod="openshift-console/console-554bb9dccb-bz2vd" Jul 09 04:39:58.266793 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:58.266579 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d99c210e-193f-416e-9c5e-234f96d978cc-console-oauth-config\") pod \"console-554bb9dccb-bz2vd\" (UID: \"d99c210e-193f-416e-9c5e-234f96d978cc\") " pod="openshift-console/console-554bb9dccb-bz2vd" Jul 09 04:39:58.266793 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:58.266601 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8m2w\" (UniqueName: \"kubernetes.io/projected/d99c210e-193f-416e-9c5e-234f96d978cc-kube-api-access-g8m2w\") pod \"console-554bb9dccb-bz2vd\" (UID: \"d99c210e-193f-416e-9c5e-234f96d978cc\") " pod="openshift-console/console-554bb9dccb-bz2vd" Jul 09 04:39:58.266793 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:58.266634 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d99c210e-193f-416e-9c5e-234f96d978cc-service-ca\") pod \"console-554bb9dccb-bz2vd\" (UID: \"d99c210e-193f-416e-9c5e-234f96d978cc\") " pod="openshift-console/console-554bb9dccb-bz2vd" Jul 09 04:39:58.266793 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:58.266681 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d99c210e-193f-416e-9c5e-234f96d978cc-oauth-serving-cert\") pod \"console-554bb9dccb-bz2vd\" (UID: \"d99c210e-193f-416e-9c5e-234f96d978cc\") " pod="openshift-console/console-554bb9dccb-bz2vd" Jul 09 04:39:58.266793 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:58.266721 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d99c210e-193f-416e-9c5e-234f96d978cc-console-serving-cert\") pod \"console-554bb9dccb-bz2vd\" (UID: \"d99c210e-193f-416e-9c5e-234f96d978cc\") " pod="openshift-console/console-554bb9dccb-bz2vd" Jul 09 04:39:58.267415 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:58.267391 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d99c210e-193f-416e-9c5e-234f96d978cc-service-ca\") pod \"console-554bb9dccb-bz2vd\" (UID: \"d99c210e-193f-416e-9c5e-234f96d978cc\") " pod="openshift-console/console-554bb9dccb-bz2vd" Jul 09 04:39:58.267525 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:58.267415 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d99c210e-193f-416e-9c5e-234f96d978cc-oauth-serving-cert\") pod \"console-554bb9dccb-bz2vd\" (UID: \"d99c210e-193f-416e-9c5e-234f96d978cc\") " pod="openshift-console/console-554bb9dccb-bz2vd" Jul 09 04:39:58.267525 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:58.267434 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d99c210e-193f-416e-9c5e-234f96d978cc-console-config\") pod \"console-554bb9dccb-bz2vd\" (UID: \"d99c210e-193f-416e-9c5e-234f96d978cc\") " pod="openshift-console/console-554bb9dccb-bz2vd" Jul 09 04:39:58.267805 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:58.267783 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d99c210e-193f-416e-9c5e-234f96d978cc-trusted-ca-bundle\") pod \"console-554bb9dccb-bz2vd\" (UID: \"d99c210e-193f-416e-9c5e-234f96d978cc\") " pod="openshift-console/console-554bb9dccb-bz2vd" Jul 09 04:39:58.269874 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:58.269851 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d99c210e-193f-416e-9c5e-234f96d978cc-console-oauth-config\") pod \"console-554bb9dccb-bz2vd\" (UID: \"d99c210e-193f-416e-9c5e-234f96d978cc\") " pod="openshift-console/console-554bb9dccb-bz2vd" Jul 09 04:39:58.269946 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:58.269933 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d99c210e-193f-416e-9c5e-234f96d978cc-console-serving-cert\") pod \"console-554bb9dccb-bz2vd\" (UID: \"d99c210e-193f-416e-9c5e-234f96d978cc\") " pod="openshift-console/console-554bb9dccb-bz2vd" Jul 09 04:39:58.274162 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:58.274144 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8m2w\" (UniqueName: \"kubernetes.io/projected/d99c210e-193f-416e-9c5e-234f96d978cc-kube-api-access-g8m2w\") pod \"console-554bb9dccb-bz2vd\" (UID: \"d99c210e-193f-416e-9c5e-234f96d978cc\") " pod="openshift-console/console-554bb9dccb-bz2vd" Jul 09 04:39:58.284714 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:58.284684 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:39:58.383975 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:58.383859 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:39:58.393662 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:58.393627 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-554bb9dccb-bz2vd" Jul 09 04:39:58.541321 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:58.541293 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-554bb9dccb-bz2vd"] Jul 09 04:39:58.544676 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:39:58.544645 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd99c210e_193f_416e_9c5e_234f96d978cc.slice/crio-f1e7809e4bbc6220d9896a492a433d59b9ac09214cd55a4d1918190a557a72e3 WatchSource:0}: Error finding container f1e7809e4bbc6220d9896a492a433d59b9ac09214cd55a4d1918190a557a72e3: Status 404 returned error can't find the container with id f1e7809e4bbc6220d9896a492a433d59b9ac09214cd55a4d1918190a557a72e3 Jul 09 04:39:58.875956 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:58.875900 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:39:58.924144 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:58.924113 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:39:59.125112 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:59.125077 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-554bb9dccb-bz2vd" event={"ID":"d99c210e-193f-416e-9c5e-234f96d978cc","Type":"ContainerStarted","Data":"4f85994d78e509e9f9e8ca021b8473d4f980f7bdc0f7790b1f79c84cb4035d07"} Jul 09 04:39:59.125288 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:59.125121 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-554bb9dccb-bz2vd" event={"ID":"d99c210e-193f-416e-9c5e-234f96d978cc","Type":"ContainerStarted","Data":"f1e7809e4bbc6220d9896a492a433d59b9ac09214cd55a4d1918190a557a72e3"} Jul 09 04:39:59.141329 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:39:59.141239 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-554bb9dccb-bz2vd" podStartSLOduration=1.141224555 podStartE2EDuration="1.141224555s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:39:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:39:59.140022921 +0000 UTC m=+471.391546499" watchObservedRunningTime="2026-07-09 04:39:59.141224555 +0000 UTC m=+471.392748132" Jul 09 04:40:00.380309 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:00.380270 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vrv4f"] Jul 09 04:40:00.383667 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:00.383648 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vrv4f" Jul 09 04:40:00.386747 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:00.386726 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-g2mdr\"" Jul 09 04:40:00.400972 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:00.400946 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vrv4f"] Jul 09 04:40:00.485468 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:00.485423 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdjb6\" (UniqueName: \"kubernetes.io/projected/2cb0aab2-be86-4b85-bb11-ba1f5dacefdb-kube-api-access-hdjb6\") pod \"limitador-operator-controller-manager-859d7b78bd-vrv4f\" (UID: \"2cb0aab2-be86-4b85-bb11-ba1f5dacefdb\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vrv4f" Jul 09 04:40:00.586043 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:00.586008 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdjb6\" (UniqueName: \"kubernetes.io/projected/2cb0aab2-be86-4b85-bb11-ba1f5dacefdb-kube-api-access-hdjb6\") pod \"limitador-operator-controller-manager-859d7b78bd-vrv4f\" (UID: \"2cb0aab2-be86-4b85-bb11-ba1f5dacefdb\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vrv4f" Jul 09 04:40:00.597221 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:00.597194 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdjb6\" (UniqueName: \"kubernetes.io/projected/2cb0aab2-be86-4b85-bb11-ba1f5dacefdb-kube-api-access-hdjb6\") pod \"limitador-operator-controller-manager-859d7b78bd-vrv4f\" (UID: \"2cb0aab2-be86-4b85-bb11-ba1f5dacefdb\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vrv4f" Jul 09 04:40:00.693681 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:00.693644 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vrv4f" Jul 09 04:40:00.828545 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:00.828521 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vrv4f"] Jul 09 04:40:00.831218 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:40:00.831192 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cb0aab2_be86_4b85_bb11_ba1f5dacefdb.slice/crio-617c2d3082204bf44919d5164e74cd638608fd55fc30ec785a89ee6244ccaaab WatchSource:0}: Error finding container 617c2d3082204bf44919d5164e74cd638608fd55fc30ec785a89ee6244ccaaab: Status 404 returned error can't find the container with id 617c2d3082204bf44919d5164e74cd638608fd55fc30ec785a89ee6244ccaaab Jul 09 04:40:01.135491 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:01.135395 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vrv4f" event={"ID":"2cb0aab2-be86-4b85-bb11-ba1f5dacefdb","Type":"ContainerStarted","Data":"617c2d3082204bf44919d5164e74cd638608fd55fc30ec785a89ee6244ccaaab"} Jul 09 04:40:01.951280 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:01.951248 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:40:03.145151 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:03.145054 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vrv4f" event={"ID":"2cb0aab2-be86-4b85-bb11-ba1f5dacefdb","Type":"ContainerStarted","Data":"b457e9ea160b6cbe328504eb5b330e8fca9f330c3e89f3995d0752a3f672c2ae"} Jul 09 04:40:03.145551 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:03.145185 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vrv4f" Jul 09 04:40:03.166156 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:03.166105 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vrv4f" podStartSLOduration=1.32672148 podStartE2EDuration="3.166089502s" totalImagesPullingTime="1.839368022s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:40:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:40:03.163974821 +0000 UTC m=+475.415498400" watchObservedRunningTime="2026-07-09 04:40:03.166089502 +0000 UTC m=+475.417613080" Jul 09 04:40:03.808607 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:03.808573 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:40:05.364657 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:05.364624 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-p6wwc"] Jul 09 04:40:05.367935 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:05.367914 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p6wwc" Jul 09 04:40:05.370583 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:05.370563 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 09 04:40:05.370702 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:05.370566 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-wgzzc\"" Jul 09 04:40:05.378130 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:05.378108 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-p6wwc"] Jul 09 04:40:05.427805 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:05.427770 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5k2p\" (UniqueName: \"kubernetes.io/projected/3def7d50-d793-4f7b-bdd9-6c815bfe6b98-kube-api-access-h5k2p\") pod \"dns-operator-controller-manager-864dd78c4c-p6wwc\" (UID: \"3def7d50-d793-4f7b-bdd9-6c815bfe6b98\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p6wwc" Jul 09 04:40:05.528367 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:05.528302 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5k2p\" (UniqueName: \"kubernetes.io/projected/3def7d50-d793-4f7b-bdd9-6c815bfe6b98-kube-api-access-h5k2p\") pod \"dns-operator-controller-manager-864dd78c4c-p6wwc\" (UID: \"3def7d50-d793-4f7b-bdd9-6c815bfe6b98\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p6wwc" Jul 09 04:40:05.536417 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:05.536386 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5k2p\" (UniqueName: \"kubernetes.io/projected/3def7d50-d793-4f7b-bdd9-6c815bfe6b98-kube-api-access-h5k2p\") pod \"dns-operator-controller-manager-864dd78c4c-p6wwc\" (UID: \"3def7d50-d793-4f7b-bdd9-6c815bfe6b98\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p6wwc" Jul 09 04:40:05.679032 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:05.678997 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p6wwc" Jul 09 04:40:05.819118 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:05.819090 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-p6wwc"] Jul 09 04:40:05.985011 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:05.984903 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:40:06.158903 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:06.158868 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p6wwc" event={"ID":"3def7d50-d793-4f7b-bdd9-6c815bfe6b98","Type":"ContainerStarted","Data":"732527e4b72e2fec9d50d5efdc4b733af6e7cabdb7ea9bfee5736016fbf59aaf"} Jul 09 04:40:06.737847 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:06.737813 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:40:08.228472 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:08.228452 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 09 04:40:08.394601 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:08.394510 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-554bb9dccb-bz2vd" Jul 09 04:40:08.394754 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:08.394619 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-554bb9dccb-bz2vd" Jul 09 04:40:08.399680 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:08.399653 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-554bb9dccb-bz2vd" Jul 09 04:40:09.105598 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:09.105559 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-jmqxw"] Jul 09 04:40:09.109100 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:09.109072 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jmqxw" Jul 09 04:40:09.111781 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:09.111761 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-dkhl9\"" Jul 09 04:40:09.123047 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:09.123024 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-jmqxw"] Jul 09 04:40:09.161682 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:09.161651 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjx66\" (UniqueName: \"kubernetes.io/projected/1d55e992-8593-4288-b65c-29adcb45c42a-kube-api-access-gjx66\") pod \"authorino-operator-75cc8d5cb7-jmqxw\" (UID: \"1d55e992-8593-4288-b65c-29adcb45c42a\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-jmqxw" Jul 09 04:40:09.174953 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:09.174918 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p6wwc" event={"ID":"3def7d50-d793-4f7b-bdd9-6c815bfe6b98","Type":"ContainerStarted","Data":"6dadbbc11d61db06ef2a3d72fd0a9a54b6ac3ba793ea9cd7577c06764c624736"} Jul 09 04:40:09.178831 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:09.178808 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-554bb9dccb-bz2vd" Jul 09 04:40:09.199653 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:09.199604 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p6wwc" podStartSLOduration=1.794291763 podStartE2EDuration="4.199586108s" totalImagesPullingTime="2.405294345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:40:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:40:09.198080002 +0000 UTC m=+481.449603581" watchObservedRunningTime="2026-07-09 04:40:09.199586108 +0000 UTC m=+481.451109754" Jul 09 04:40:09.262563 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:09.262179 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjx66\" (UniqueName: \"kubernetes.io/projected/1d55e992-8593-4288-b65c-29adcb45c42a-kube-api-access-gjx66\") pod \"authorino-operator-75cc8d5cb7-jmqxw\" (UID: \"1d55e992-8593-4288-b65c-29adcb45c42a\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-jmqxw" Jul 09 04:40:09.288314 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:09.288282 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjx66\" (UniqueName: \"kubernetes.io/projected/1d55e992-8593-4288-b65c-29adcb45c42a-kube-api-access-gjx66\") pod \"authorino-operator-75cc8d5cb7-jmqxw\" (UID: \"1d55e992-8593-4288-b65c-29adcb45c42a\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-jmqxw" Jul 09 04:40:09.294429 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:09.294400 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c845bc447-d42qw"] Jul 09 04:40:09.419882 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:09.419852 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jmqxw" Jul 09 04:40:09.436598 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:09.432684 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:40:09.535105 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:09.535048 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:40:09.556461 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:09.556440 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-jmqxw"] Jul 09 04:40:09.559241 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:40:09.559213 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d55e992_8593_4288_b65c_29adcb45c42a.slice/crio-c823551430fc1d07d31676c652c548b122675fbb2c35345884a4326914cfc10f WatchSource:0}: Error finding container c823551430fc1d07d31676c652c548b122675fbb2c35345884a4326914cfc10f: Status 404 returned error can't find the container with id c823551430fc1d07d31676c652c548b122675fbb2c35345884a4326914cfc10f Jul 09 04:40:09.635646 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:09.635585 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:40:09.736166 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:09.736074 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:40:09.944787 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:09.944759 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:40:10.180214 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:10.180180 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jmqxw" event={"ID":"1d55e992-8593-4288-b65c-29adcb45c42a","Type":"ContainerStarted","Data":"c823551430fc1d07d31676c652c548b122675fbb2c35345884a4326914cfc10f"} Jul 09 04:40:10.180520 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:10.180503 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p6wwc" Jul 09 04:40:12.195893 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:12.195853 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jmqxw" event={"ID":"1d55e992-8593-4288-b65c-29adcb45c42a","Type":"ContainerStarted","Data":"bf50017df03304653ae8a1288e0e7000b51af00901625e86ebd173ee1efe01ee"} Jul 09 04:40:12.196262 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:12.195913 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jmqxw" Jul 09 04:40:12.209102 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:12.209058 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jmqxw" podStartSLOduration=0.878902509 podStartE2EDuration="3.209039995s" totalImagesPullingTime="2.330137486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:40:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:40:12.20854499 +0000 UTC m=+484.460068569" watchObservedRunningTime="2026-07-09 04:40:12.209039995 +0000 UTC m=+484.460563573" Jul 09 04:40:12.618573 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:12.618497 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:40:12.710697 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:12.710662 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:40:14.151930 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:14.151905 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vrv4f" Jul 09 04:40:15.006984 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:15.006924 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-24vcn"] Jul 09 04:40:15.010238 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:15.010215 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-24vcn" Jul 09 04:40:15.014083 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:15.014055 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-kp4rx\"" Jul 09 04:40:15.034440 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:15.034414 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-24vcn"] Jul 09 04:40:15.094560 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:15.094528 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:40:15.111187 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:15.111160 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnjf5\" (UniqueName: \"kubernetes.io/projected/807c8d27-3abe-482a-967b-ba3b66a83d17-kube-api-access-nnjf5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-24vcn\" (UID: \"807c8d27-3abe-482a-967b-ba3b66a83d17\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-24vcn" Jul 09 04:40:15.111307 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:15.111198 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/807c8d27-3abe-482a-967b-ba3b66a83d17-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-24vcn\" (UID: \"807c8d27-3abe-482a-967b-ba3b66a83d17\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-24vcn" Jul 09 04:40:15.211672 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:15.211630 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnjf5\" (UniqueName: \"kubernetes.io/projected/807c8d27-3abe-482a-967b-ba3b66a83d17-kube-api-access-nnjf5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-24vcn\" (UID: \"807c8d27-3abe-482a-967b-ba3b66a83d17\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-24vcn" Jul 09 04:40:15.212027 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:15.211682 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/807c8d27-3abe-482a-967b-ba3b66a83d17-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-24vcn\" (UID: \"807c8d27-3abe-482a-967b-ba3b66a83d17\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-24vcn" Jul 09 04:40:15.212027 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:15.212013 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/807c8d27-3abe-482a-967b-ba3b66a83d17-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-24vcn\" (UID: \"807c8d27-3abe-482a-967b-ba3b66a83d17\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-24vcn" Jul 09 04:40:15.219826 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:15.219802 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnjf5\" (UniqueName: \"kubernetes.io/projected/807c8d27-3abe-482a-967b-ba3b66a83d17-kube-api-access-nnjf5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-24vcn\" (UID: \"807c8d27-3abe-482a-967b-ba3b66a83d17\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-24vcn" Jul 09 04:40:15.320226 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:15.320130 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-24vcn" Jul 09 04:40:15.477416 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:15.477378 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-24vcn"] Jul 09 04:40:15.479383 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:40:15.479358 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod807c8d27_3abe_482a_967b_ba3b66a83d17.slice/crio-1c99eac76301f971a5a147acc084c57be50f7188d77ef5c04e6c053ba7a73dcb WatchSource:0}: Error finding container 1c99eac76301f971a5a147acc084c57be50f7188d77ef5c04e6c053ba7a73dcb: Status 404 returned error can't find the container with id 1c99eac76301f971a5a147acc084c57be50f7188d77ef5c04e6c053ba7a73dcb Jul 09 04:40:16.211252 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:16.211202 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-24vcn" event={"ID":"807c8d27-3abe-482a-967b-ba3b66a83d17","Type":"ContainerStarted","Data":"1c99eac76301f971a5a147acc084c57be50f7188d77ef5c04e6c053ba7a73dcb"} Jul 09 04:40:16.815609 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:16.815574 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:40:18.980530 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:18.980485 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:40:21.188857 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:21.188820 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p6wwc" Jul 09 04:40:21.578521 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:21.578427 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:40:21.671721 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:21.671667 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:40:22.240810 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:22.240778 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-24vcn" event={"ID":"807c8d27-3abe-482a-967b-ba3b66a83d17","Type":"ContainerStarted","Data":"bd89f76b9ca497afc2bc6f33f434a44017705e6cc7bdacb117f0b488c054cbd6"} Jul 09 04:40:22.241230 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:22.241016 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-24vcn" Jul 09 04:40:22.261428 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:22.261371 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-24vcn" podStartSLOduration=1.96155365 podStartE2EDuration="8.261353304s" totalImagesPullingTime="6.299799654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:40:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:40:22.259572483 +0000 UTC m=+494.511096106" watchObservedRunningTime="2026-07-09 04:40:22.261353304 +0000 UTC m=+494.512876875" Jul 09 04:40:23.198706 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:23.198672 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-jmqxw" Jul 09 04:40:24.625953 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:24.625925 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-fw8fm"] Jul 09 04:40:24.629263 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:24.629243 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fw8fm" Jul 09 04:40:24.629796 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:24.629771 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:40:24.631745 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:24.631724 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 09 04:40:24.631835 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:24.631728 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 09 04:40:24.631835 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:24.631728 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-ckvp2\"" Jul 09 04:40:24.636551 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:24.636527 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-fw8fm"] Jul 09 04:40:24.697314 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:24.697275 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgvb2\" (UniqueName: \"kubernetes.io/projected/c57010ee-79c3-4870-b65b-c5408ab24ef7-kube-api-access-qgvb2\") pod \"kuadrant-console-plugin-66b5774985-fw8fm\" (UID: \"c57010ee-79c3-4870-b65b-c5408ab24ef7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fw8fm" Jul 09 04:40:24.697314 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:24.697315 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c57010ee-79c3-4870-b65b-c5408ab24ef7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-fw8fm\" (UID: \"c57010ee-79c3-4870-b65b-c5408ab24ef7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fw8fm" Jul 09 04:40:24.697574 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:24.697462 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c57010ee-79c3-4870-b65b-c5408ab24ef7-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-fw8fm\" (UID: \"c57010ee-79c3-4870-b65b-c5408ab24ef7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fw8fm" Jul 09 04:40:24.798051 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:24.798013 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgvb2\" (UniqueName: \"kubernetes.io/projected/c57010ee-79c3-4870-b65b-c5408ab24ef7-kube-api-access-qgvb2\") pod \"kuadrant-console-plugin-66b5774985-fw8fm\" (UID: \"c57010ee-79c3-4870-b65b-c5408ab24ef7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fw8fm" Jul 09 04:40:24.798051 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:24.798054 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c57010ee-79c3-4870-b65b-c5408ab24ef7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-fw8fm\" (UID: \"c57010ee-79c3-4870-b65b-c5408ab24ef7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fw8fm" Jul 09 04:40:24.798267 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:24.798099 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c57010ee-79c3-4870-b65b-c5408ab24ef7-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-fw8fm\" (UID: \"c57010ee-79c3-4870-b65b-c5408ab24ef7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fw8fm" Jul 09 04:40:24.798267 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:40:24.798222 2565 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 09 04:40:24.798377 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:40:24.798284 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c57010ee-79c3-4870-b65b-c5408ab24ef7-plugin-serving-cert podName:c57010ee-79c3-4870-b65b-c5408ab24ef7 nodeName:}" failed. No retries permitted until 2026-07-09 04:40:25.298267649 +0000 UTC m=+497.549791205 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/c57010ee-79c3-4870-b65b-c5408ab24ef7-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-fw8fm" (UID: "c57010ee-79c3-4870-b65b-c5408ab24ef7") : secret "plugin-serving-cert" not found Jul 09 04:40:24.798724 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:24.798706 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c57010ee-79c3-4870-b65b-c5408ab24ef7-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-fw8fm\" (UID: \"c57010ee-79c3-4870-b65b-c5408ab24ef7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fw8fm" Jul 09 04:40:24.805942 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:24.805912 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgvb2\" (UniqueName: \"kubernetes.io/projected/c57010ee-79c3-4870-b65b-c5408ab24ef7-kube-api-access-qgvb2\") pod \"kuadrant-console-plugin-66b5774985-fw8fm\" (UID: \"c57010ee-79c3-4870-b65b-c5408ab24ef7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fw8fm" Jul 09 04:40:25.302297 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:25.302257 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c57010ee-79c3-4870-b65b-c5408ab24ef7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-fw8fm\" (UID: \"c57010ee-79c3-4870-b65b-c5408ab24ef7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fw8fm" Jul 09 04:40:25.302517 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:40:25.302427 2565 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 09 04:40:25.302517 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:40:25.302488 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c57010ee-79c3-4870-b65b-c5408ab24ef7-plugin-serving-cert podName:c57010ee-79c3-4870-b65b-c5408ab24ef7 nodeName:}" failed. No retries permitted until 2026-07-09 04:40:26.302474115 +0000 UTC m=+498.553997671 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/c57010ee-79c3-4870-b65b-c5408ab24ef7-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-fw8fm" (UID: "c57010ee-79c3-4870-b65b-c5408ab24ef7") : secret "plugin-serving-cert" not found Jul 09 04:40:25.373193 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:25.373159 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:40:26.312155 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:26.312110 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c57010ee-79c3-4870-b65b-c5408ab24ef7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-fw8fm\" (UID: \"c57010ee-79c3-4870-b65b-c5408ab24ef7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fw8fm" Jul 09 04:40:26.314582 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:26.314555 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c57010ee-79c3-4870-b65b-c5408ab24ef7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-fw8fm\" (UID: \"c57010ee-79c3-4870-b65b-c5408ab24ef7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fw8fm" Jul 09 04:40:26.439758 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:26.439710 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fw8fm" Jul 09 04:40:26.567955 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:26.567929 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-fw8fm"] Jul 09 04:40:26.569924 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:40:26.569887 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc57010ee_79c3_4870_b65b_c5408ab24ef7.slice/crio-6883f5d07c3d7ae3dee60d2db615a5f5054482b69269bed6a1c43a4e35d3dd1d WatchSource:0}: Error finding container 6883f5d07c3d7ae3dee60d2db615a5f5054482b69269bed6a1c43a4e35d3dd1d: Status 404 returned error can't find the container with id 6883f5d07c3d7ae3dee60d2db615a5f5054482b69269bed6a1c43a4e35d3dd1d Jul 09 04:40:27.266770 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:27.266739 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fw8fm" event={"ID":"c57010ee-79c3-4870-b65b-c5408ab24ef7","Type":"ContainerStarted","Data":"6883f5d07c3d7ae3dee60d2db615a5f5054482b69269bed6a1c43a4e35d3dd1d"} Jul 09 04:40:27.346890 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:27.346841 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:40:27.396382 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:27.396332 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:40:27.948968 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:27.948931 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:40:28.051082 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:28.051012 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:40:32.150203 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:32.150169 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:40:32.250373 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:32.250277 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:40:33.248488 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:33.248460 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-24vcn" Jul 09 04:40:34.314850 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:34.314809 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6c845bc447-d42qw" podUID="680a9c76-039a-4113-bd5e-ff063c22bcdf" containerName="console" containerID="cri-o://608d9ba7bcbafe28adead8db7033e531241f4268ad3b026a45279b97685fb61d" gracePeriod=15 Jul 09 04:40:34.938546 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:34.938510 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-24vcn"] Jul 09 04:40:34.938762 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:34.938735 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-24vcn" podUID="807c8d27-3abe-482a-967b-ba3b66a83d17" containerName="manager" containerID="cri-o://bd89f76b9ca497afc2bc6f33f434a44017705e6cc7bdacb117f0b488c054cbd6" gracePeriod=2 Jul 09 04:40:34.953699 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:34.953634 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-24vcn"] Jul 09 04:40:34.960965 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:40:34.960922 2565 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"807c8d27-3abe-482a-967b-ba3b66a83d17\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-24vcn\": pods \"kuadrant-operator-controller-manager-8684c9ff49-24vcn\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-24vcn" Jul 09 04:40:34.974271 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:34.974120 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vrv4f"] Jul 09 04:40:34.974451 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:34.974381 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vrv4f" podUID="2cb0aab2-be86-4b85-bb11-ba1f5dacefdb" containerName="manager" containerID="cri-o://b457e9ea160b6cbe328504eb5b330e8fca9f330c3e89f3995d0752a3f672c2ae" gracePeriod=2 Jul 09 04:40:35.000470 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:35.000323 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jlmh5"] Jul 09 04:40:35.000961 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:35.000910 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="807c8d27-3abe-482a-967b-ba3b66a83d17" containerName="manager" Jul 09 04:40:35.000961 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:35.000934 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="807c8d27-3abe-482a-967b-ba3b66a83d17" containerName="manager" Jul 09 04:40:35.001201 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:35.001044 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="807c8d27-3abe-482a-967b-ba3b66a83d17" containerName="manager" Jul 09 04:40:35.005901 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:35.005873 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vrv4f"] Jul 09 04:40:35.006048 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:35.005970 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jlmh5" Jul 09 04:40:35.007473 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:40:35.007429 2565 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"2cb0aab2-be86-4b85-bb11-ba1f5dacefdb\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-vrv4f\": pods \"limitador-operator-controller-manager-859d7b78bd-vrv4f\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vrv4f" Jul 09 04:40:35.010923 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:40:35.010874 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-24vcn\" is forbidden: User \"system:node:ip-10-0-139-184.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-184.ec2.internal' and this object" podUID="807c8d27-3abe-482a-967b-ba3b66a83d17" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-24vcn" Jul 09 04:40:35.013174 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:40:35.013145 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-24vcn\" is forbidden: User \"system:node:ip-10-0-139-184.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-184.ec2.internal' and this object" podUID="807c8d27-3abe-482a-967b-ba3b66a83d17" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-24vcn" Jul 09 04:40:35.016392 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:35.016365 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zzrz7"] Jul 09 04:40:35.016890 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:35.016871 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cb0aab2-be86-4b85-bb11-ba1f5dacefdb" containerName="manager" Jul 09 04:40:35.016970 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:35.016894 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cb0aab2-be86-4b85-bb11-ba1f5dacefdb" containerName="manager" Jul 09 04:40:35.017020 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:35.016996 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cb0aab2-be86-4b85-bb11-ba1f5dacefdb" containerName="manager" Jul 09 04:40:35.020093 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:35.020073 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zzrz7" Jul 09 04:40:35.027278 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:35.027171 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jlmh5"] Jul 09 04:40:35.036272 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:35.036237 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zzrz7"] Jul 09 04:40:35.049472 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:40:35.049221 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vrv4f\" is forbidden: User \"system:node:ip-10-0-139-184.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-184.ec2.internal' and this object" podUID="2cb0aab2-be86-4b85-bb11-ba1f5dacefdb" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vrv4f" Jul 09 04:40:35.055427 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:40:35.055375 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-24vcn\" is forbidden: User \"system:node:ip-10-0-139-184.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-184.ec2.internal' and this object" podUID="807c8d27-3abe-482a-967b-ba3b66a83d17" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-24vcn" Jul 09 04:40:35.057644 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:40:35.057607 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vrv4f\" is forbidden: User \"system:node:ip-10-0-139-184.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-184.ec2.internal' and this object" podUID="2cb0aab2-be86-4b85-bb11-ba1f5dacefdb" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vrv4f" Jul 09 04:40:35.094451 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:35.094415 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/098d6f62-82ee-4778-a992-83e0f4914889-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jlmh5\" (UID: \"098d6f62-82ee-4778-a992-83e0f4914889\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jlmh5" Jul 09 04:40:35.094654 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:35.094498 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w58tb\" (UniqueName: \"kubernetes.io/projected/098d6f62-82ee-4778-a992-83e0f4914889-kube-api-access-w58tb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jlmh5\" (UID: \"098d6f62-82ee-4778-a992-83e0f4914889\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jlmh5" Jul 09 04:40:35.195217 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:35.195137 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/098d6f62-82ee-4778-a992-83e0f4914889-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jlmh5\" (UID: \"098d6f62-82ee-4778-a992-83e0f4914889\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jlmh5" Jul 09 04:40:35.195217 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:35.195192 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cwlt\" (UniqueName: \"kubernetes.io/projected/514ee074-905b-413c-b139-aec3fabc856c-kube-api-access-5cwlt\") pod \"limitador-operator-controller-manager-859d7b78bd-zzrz7\" (UID: \"514ee074-905b-413c-b139-aec3fabc856c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zzrz7" Jul 09 04:40:35.195447 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:35.195257 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w58tb\" (UniqueName: \"kubernetes.io/projected/098d6f62-82ee-4778-a992-83e0f4914889-kube-api-access-w58tb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jlmh5\" (UID: \"098d6f62-82ee-4778-a992-83e0f4914889\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jlmh5" Jul 09 04:40:35.195627 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:35.195604 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/098d6f62-82ee-4778-a992-83e0f4914889-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jlmh5\" (UID: \"098d6f62-82ee-4778-a992-83e0f4914889\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jlmh5" Jul 09 04:40:35.210771 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:35.210733 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w58tb\" (UniqueName: \"kubernetes.io/projected/098d6f62-82ee-4778-a992-83e0f4914889-kube-api-access-w58tb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jlmh5\" (UID: \"098d6f62-82ee-4778-a992-83e0f4914889\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jlmh5" Jul 09 04:40:35.296794 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:35.296752 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cwlt\" (UniqueName: \"kubernetes.io/projected/514ee074-905b-413c-b139-aec3fabc856c-kube-api-access-5cwlt\") pod \"limitador-operator-controller-manager-859d7b78bd-zzrz7\" (UID: \"514ee074-905b-413c-b139-aec3fabc856c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zzrz7" Jul 09 04:40:35.309529 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:35.309499 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cwlt\" (UniqueName: \"kubernetes.io/projected/514ee074-905b-413c-b139-aec3fabc856c-kube-api-access-5cwlt\") pod \"limitador-operator-controller-manager-859d7b78bd-zzrz7\" (UID: \"514ee074-905b-413c-b139-aec3fabc856c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zzrz7" Jul 09 04:40:35.311258 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:35.311227 2565 generic.go:356] "Generic (PLEG): container finished" podID="680a9c76-039a-4113-bd5e-ff063c22bcdf" containerID="608d9ba7bcbafe28adead8db7033e531241f4268ad3b026a45279b97685fb61d" exitCode=0 Jul 09 04:40:35.311430 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:35.311305 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c845bc447-d42qw" event={"ID":"680a9c76-039a-4113-bd5e-ff063c22bcdf","Type":"ContainerDied","Data":"608d9ba7bcbafe28adead8db7033e531241f4268ad3b026a45279b97685fb61d"} Jul 09 04:40:35.353460 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:35.353425 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jlmh5" Jul 09 04:40:35.361882 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:35.361850 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zzrz7" Jul 09 04:40:36.236946 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:36.236895 2565 patch_prober.go:28] interesting pod/console-6c845bc447-d42qw container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.133.0.16:8443/health\": context deadline exceeded" start-of-body= Jul 09 04:40:36.237129 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:36.236971 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-6c845bc447-d42qw" podUID="680a9c76-039a-4113-bd5e-ff063c22bcdf" containerName="console" probeResult="failure" output="Get \"https://10.133.0.16:8443/health\": context deadline exceeded" Jul 09 04:40:41.463234 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:41.463198 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:40:41.566498 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:41.566462 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:40:43.968457 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:43.968422 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:40:44.022196 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:44.022162 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:40:46.237414 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:46.237374 2565 patch_prober.go:28] interesting pod/console-6c845bc447-d42qw container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.133.0.16:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 09 04:40:46.237959 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:46.237439 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-6c845bc447-d42qw" podUID="680a9c76-039a-4113-bd5e-ff063c22bcdf" containerName="console" probeResult="failure" output="Get \"https://10.133.0.16:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Jul 09 04:40:48.684554 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:48.684523 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c845bc447-d42qw" Jul 09 04:40:48.723812 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:48.720766 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/680a9c76-039a-4113-bd5e-ff063c22bcdf-console-oauth-config\") pod \"680a9c76-039a-4113-bd5e-ff063c22bcdf\" (UID: \"680a9c76-039a-4113-bd5e-ff063c22bcdf\") " Jul 09 04:40:48.723812 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:48.720850 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/680a9c76-039a-4113-bd5e-ff063c22bcdf-console-serving-cert\") pod \"680a9c76-039a-4113-bd5e-ff063c22bcdf\" (UID: \"680a9c76-039a-4113-bd5e-ff063c22bcdf\") " Jul 09 04:40:48.723812 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:48.720905 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/680a9c76-039a-4113-bd5e-ff063c22bcdf-trusted-ca-bundle\") pod \"680a9c76-039a-4113-bd5e-ff063c22bcdf\" (UID: \"680a9c76-039a-4113-bd5e-ff063c22bcdf\") " Jul 09 04:40:48.723812 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:48.720936 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/680a9c76-039a-4113-bd5e-ff063c22bcdf-console-config\") pod \"680a9c76-039a-4113-bd5e-ff063c22bcdf\" (UID: \"680a9c76-039a-4113-bd5e-ff063c22bcdf\") " Jul 09 04:40:48.723812 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:48.720961 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/680a9c76-039a-4113-bd5e-ff063c22bcdf-service-ca\") pod \"680a9c76-039a-4113-bd5e-ff063c22bcdf\" (UID: \"680a9c76-039a-4113-bd5e-ff063c22bcdf\") " Jul 09 04:40:48.723812 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:48.720996 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/680a9c76-039a-4113-bd5e-ff063c22bcdf-oauth-serving-cert\") pod \"680a9c76-039a-4113-bd5e-ff063c22bcdf\" (UID: \"680a9c76-039a-4113-bd5e-ff063c22bcdf\") " Jul 09 04:40:48.723812 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:48.721062 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tg2xq\" (UniqueName: \"kubernetes.io/projected/680a9c76-039a-4113-bd5e-ff063c22bcdf-kube-api-access-tg2xq\") pod \"680a9c76-039a-4113-bd5e-ff063c22bcdf\" (UID: \"680a9c76-039a-4113-bd5e-ff063c22bcdf\") " Jul 09 04:40:48.723812 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:48.722613 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/680a9c76-039a-4113-bd5e-ff063c22bcdf-service-ca" (OuterVolumeSpecName: "service-ca") pod "680a9c76-039a-4113-bd5e-ff063c22bcdf" (UID: "680a9c76-039a-4113-bd5e-ff063c22bcdf"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:40:48.723812 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:48.723025 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/680a9c76-039a-4113-bd5e-ff063c22bcdf-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "680a9c76-039a-4113-bd5e-ff063c22bcdf" (UID: "680a9c76-039a-4113-bd5e-ff063c22bcdf"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:40:48.723812 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:48.723417 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/680a9c76-039a-4113-bd5e-ff063c22bcdf-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "680a9c76-039a-4113-bd5e-ff063c22bcdf" (UID: "680a9c76-039a-4113-bd5e-ff063c22bcdf"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:40:48.723812 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:48.723764 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/680a9c76-039a-4113-bd5e-ff063c22bcdf-console-config" (OuterVolumeSpecName: "console-config") pod "680a9c76-039a-4113-bd5e-ff063c22bcdf" (UID: "680a9c76-039a-4113-bd5e-ff063c22bcdf"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:40:48.726047 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:48.726009 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/680a9c76-039a-4113-bd5e-ff063c22bcdf-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "680a9c76-039a-4113-bd5e-ff063c22bcdf" (UID: "680a9c76-039a-4113-bd5e-ff063c22bcdf"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:40:48.731099 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:48.731052 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/680a9c76-039a-4113-bd5e-ff063c22bcdf-kube-api-access-tg2xq" (OuterVolumeSpecName: "kube-api-access-tg2xq") pod "680a9c76-039a-4113-bd5e-ff063c22bcdf" (UID: "680a9c76-039a-4113-bd5e-ff063c22bcdf"). InnerVolumeSpecName "kube-api-access-tg2xq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:40:48.733097 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:48.733048 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/680a9c76-039a-4113-bd5e-ff063c22bcdf-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "680a9c76-039a-4113-bd5e-ff063c22bcdf" (UID: "680a9c76-039a-4113-bd5e-ff063c22bcdf"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:40:48.822196 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:48.822122 2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/680a9c76-039a-4113-bd5e-ff063c22bcdf-console-oauth-config\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:40:48.822196 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:48.822152 2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/680a9c76-039a-4113-bd5e-ff063c22bcdf-console-serving-cert\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:40:48.822196 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:48.822167 2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/680a9c76-039a-4113-bd5e-ff063c22bcdf-trusted-ca-bundle\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:40:48.822196 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:48.822186 2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/680a9c76-039a-4113-bd5e-ff063c22bcdf-console-config\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:40:48.822196 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:48.822200 2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/680a9c76-039a-4113-bd5e-ff063c22bcdf-service-ca\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:40:48.822808 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:48.822213 2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/680a9c76-039a-4113-bd5e-ff063c22bcdf-oauth-serving-cert\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:40:48.822808 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:48.822229 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tg2xq\" (UniqueName: \"kubernetes.io/projected/680a9c76-039a-4113-bd5e-ff063c22bcdf-kube-api-access-tg2xq\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:40:48.833539 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:48.833326 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jlmh5"] Jul 09 04:40:48.834419 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:48.833705 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-24vcn" Jul 09 04:40:48.837589 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:40:48.835902 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-24vcn\" is forbidden: User \"system:node:ip-10-0-139-184.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-184.ec2.internal' and this object" podUID="807c8d27-3abe-482a-967b-ba3b66a83d17" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-24vcn" Jul 09 04:40:48.838273 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:40:48.838229 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod098d6f62_82ee_4778_a992_83e0f4914889.slice/crio-81398295cf79caed37cbadf00b33fa88da5437f7a092c2ff4ba94c29d0de736a WatchSource:0}: Error finding container 81398295cf79caed37cbadf00b33fa88da5437f7a092c2ff4ba94c29d0de736a: Status 404 returned error can't find the container with id 81398295cf79caed37cbadf00b33fa88da5437f7a092c2ff4ba94c29d0de736a Jul 09 04:40:48.844440 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:48.844415 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vrv4f" Jul 09 04:40:48.846378 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:40:48.846331 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vrv4f\" is forbidden: User \"system:node:ip-10-0-139-184.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-184.ec2.internal' and this object" podUID="2cb0aab2-be86-4b85-bb11-ba1f5dacefdb" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vrv4f" Jul 09 04:40:48.848072 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:40:48.848047 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-24vcn\" is forbidden: User \"system:node:ip-10-0-139-184.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-184.ec2.internal' and this object" podUID="807c8d27-3abe-482a-967b-ba3b66a83d17" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-24vcn" Jul 09 04:40:48.855195 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:48.855173 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zzrz7"] Jul 09 04:40:48.859011 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:40:48.858988 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod514ee074_905b_413c_b139_aec3fabc856c.slice/crio-92e01c4c7b0f0953cf7e8295deb16463c193a8a2fe843ef8f85cece0c1585d3a WatchSource:0}: Error finding container 92e01c4c7b0f0953cf7e8295deb16463c193a8a2fe843ef8f85cece0c1585d3a: Status 404 returned error can't find the container with id 92e01c4c7b0f0953cf7e8295deb16463c193a8a2fe843ef8f85cece0c1585d3a Jul 09 04:40:48.923385 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:48.923357 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hdjb6\" (UniqueName: \"kubernetes.io/projected/2cb0aab2-be86-4b85-bb11-ba1f5dacefdb-kube-api-access-hdjb6\") pod \"2cb0aab2-be86-4b85-bb11-ba1f5dacefdb\" (UID: \"2cb0aab2-be86-4b85-bb11-ba1f5dacefdb\") " Jul 09 04:40:48.923500 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:48.923427 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nnjf5\" (UniqueName: \"kubernetes.io/projected/807c8d27-3abe-482a-967b-ba3b66a83d17-kube-api-access-nnjf5\") pod \"807c8d27-3abe-482a-967b-ba3b66a83d17\" (UID: \"807c8d27-3abe-482a-967b-ba3b66a83d17\") " Jul 09 04:40:48.923686 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:48.923577 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/807c8d27-3abe-482a-967b-ba3b66a83d17-extensions-socket-volume\") pod \"807c8d27-3abe-482a-967b-ba3b66a83d17\" (UID: \"807c8d27-3abe-482a-967b-ba3b66a83d17\") " Jul 09 04:40:48.924136 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:48.924097 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/807c8d27-3abe-482a-967b-ba3b66a83d17-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "807c8d27-3abe-482a-967b-ba3b66a83d17" (UID: "807c8d27-3abe-482a-967b-ba3b66a83d17"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:40:48.925985 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:48.925963 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/807c8d27-3abe-482a-967b-ba3b66a83d17-kube-api-access-nnjf5" (OuterVolumeSpecName: "kube-api-access-nnjf5") pod "807c8d27-3abe-482a-967b-ba3b66a83d17" (UID: "807c8d27-3abe-482a-967b-ba3b66a83d17"). InnerVolumeSpecName "kube-api-access-nnjf5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:40:48.926063 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:48.925957 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cb0aab2-be86-4b85-bb11-ba1f5dacefdb-kube-api-access-hdjb6" (OuterVolumeSpecName: "kube-api-access-hdjb6") pod "2cb0aab2-be86-4b85-bb11-ba1f5dacefdb" (UID: "2cb0aab2-be86-4b85-bb11-ba1f5dacefdb"). InnerVolumeSpecName "kube-api-access-hdjb6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:40:49.024746 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:49.024713 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nnjf5\" (UniqueName: \"kubernetes.io/projected/807c8d27-3abe-482a-967b-ba3b66a83d17-kube-api-access-nnjf5\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:40:49.024746 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:49.024742 2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/807c8d27-3abe-482a-967b-ba3b66a83d17-extensions-socket-volume\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:40:49.024746 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:49.024752 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdjb6\" (UniqueName: \"kubernetes.io/projected/2cb0aab2-be86-4b85-bb11-ba1f5dacefdb-kube-api-access-hdjb6\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:40:49.385865 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:49.385828 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zzrz7" event={"ID":"514ee074-905b-413c-b139-aec3fabc856c","Type":"ContainerStarted","Data":"e68652dbebb0e8eeb1010a9735492e040f0810202175d87af2e385fdaa257486"} Jul 09 04:40:49.386076 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:49.385885 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zzrz7" event={"ID":"514ee074-905b-413c-b139-aec3fabc856c","Type":"ContainerStarted","Data":"92e01c4c7b0f0953cf7e8295deb16463c193a8a2fe843ef8f85cece0c1585d3a"} Jul 09 04:40:49.386076 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:49.385995 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zzrz7" Jul 09 04:40:49.387534 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:49.387503 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jlmh5" event={"ID":"098d6f62-82ee-4778-a992-83e0f4914889","Type":"ContainerStarted","Data":"929efcc0dbf77cb710b0c085309bfd8f338a5a905bae0e26f19ec3fed643b5cc"} Jul 09 04:40:49.387667 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:49.387548 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jlmh5" event={"ID":"098d6f62-82ee-4778-a992-83e0f4914889","Type":"ContainerStarted","Data":"81398295cf79caed37cbadf00b33fa88da5437f7a092c2ff4ba94c29d0de736a"} Jul 09 04:40:49.387667 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:49.387573 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jlmh5" Jul 09 04:40:49.388065 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:40:49.388034 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-24vcn\" is forbidden: User \"system:node:ip-10-0-139-184.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-184.ec2.internal' and this object" podUID="807c8d27-3abe-482a-967b-ba3b66a83d17" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-24vcn" Jul 09 04:40:49.388816 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:49.388795 2565 generic.go:356] "Generic (PLEG): container finished" podID="2cb0aab2-be86-4b85-bb11-ba1f5dacefdb" containerID="b457e9ea160b6cbe328504eb5b330e8fca9f330c3e89f3995d0752a3f672c2ae" exitCode=0 Jul 09 04:40:49.388934 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:49.388826 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vrv4f" Jul 09 04:40:49.388934 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:49.388875 2565 scope.go:117] "RemoveContainer" containerID="b457e9ea160b6cbe328504eb5b330e8fca9f330c3e89f3995d0752a3f672c2ae" Jul 09 04:40:49.390095 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:40:49.390064 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vrv4f\" is forbidden: User \"system:node:ip-10-0-139-184.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-184.ec2.internal' and this object" podUID="2cb0aab2-be86-4b85-bb11-ba1f5dacefdb" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vrv4f" Jul 09 04:40:49.390283 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:49.390265 2565 generic.go:356] "Generic (PLEG): container finished" podID="807c8d27-3abe-482a-967b-ba3b66a83d17" containerID="bd89f76b9ca497afc2bc6f33f434a44017705e6cc7bdacb117f0b488c054cbd6" exitCode=0 Jul 09 04:40:49.390378 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:49.390295 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-24vcn" Jul 09 04:40:49.392219 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:49.392193 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fw8fm" event={"ID":"c57010ee-79c3-4870-b65b-c5408ab24ef7","Type":"ContainerStarted","Data":"8dd79853c1eae9cad7875723964399956903e49b540adee8073b1df6fa756c6e"} Jul 09 04:40:49.393793 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:49.393746 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c845bc447-d42qw" event={"ID":"680a9c76-039a-4113-bd5e-ff063c22bcdf","Type":"ContainerDied","Data":"39fc63dae10cb53d8fcc7f7baf124c53698ca9601fb1737d27c4ac93d432a961"} Jul 09 04:40:49.393920 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:49.393823 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c845bc447-d42qw" Jul 09 04:40:49.400870 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:49.400831 2565 scope.go:117] "RemoveContainer" containerID="b457e9ea160b6cbe328504eb5b330e8fca9f330c3e89f3995d0752a3f672c2ae" Jul 09 04:40:49.401166 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:40:49.401148 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b457e9ea160b6cbe328504eb5b330e8fca9f330c3e89f3995d0752a3f672c2ae\": container with ID starting with b457e9ea160b6cbe328504eb5b330e8fca9f330c3e89f3995d0752a3f672c2ae not found: ID does not exist" containerID="b457e9ea160b6cbe328504eb5b330e8fca9f330c3e89f3995d0752a3f672c2ae" Jul 09 04:40:49.401238 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:49.401173 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b457e9ea160b6cbe328504eb5b330e8fca9f330c3e89f3995d0752a3f672c2ae"} err="failed to get container status \"b457e9ea160b6cbe328504eb5b330e8fca9f330c3e89f3995d0752a3f672c2ae\": rpc error: code = NotFound desc = could not find container \"b457e9ea160b6cbe328504eb5b330e8fca9f330c3e89f3995d0752a3f672c2ae\": container with ID starting with b457e9ea160b6cbe328504eb5b330e8fca9f330c3e89f3995d0752a3f672c2ae not found: ID does not exist" Jul 09 04:40:49.401238 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:49.401192 2565 scope.go:117] "RemoveContainer" containerID="bd89f76b9ca497afc2bc6f33f434a44017705e6cc7bdacb117f0b488c054cbd6" Jul 09 04:40:49.407659 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:49.407614 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zzrz7" podStartSLOduration=15.407601505 podStartE2EDuration="15.407601505s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:40:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:40:49.405384656 +0000 UTC m=+521.656908235" watchObservedRunningTime="2026-07-09 04:40:49.407601505 +0000 UTC m=+521.659125077" Jul 09 04:40:49.410522 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:40:49.410497 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-24vcn\" is forbidden: User \"system:node:ip-10-0-139-184.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-184.ec2.internal' and this object" podUID="807c8d27-3abe-482a-967b-ba3b66a83d17" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-24vcn" Jul 09 04:40:49.411806 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:49.411786 2565 scope.go:117] "RemoveContainer" containerID="bd89f76b9ca497afc2bc6f33f434a44017705e6cc7bdacb117f0b488c054cbd6" Jul 09 04:40:49.412076 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:40:49.412049 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd89f76b9ca497afc2bc6f33f434a44017705e6cc7bdacb117f0b488c054cbd6\": container with ID starting with bd89f76b9ca497afc2bc6f33f434a44017705e6cc7bdacb117f0b488c054cbd6 not found: ID does not exist" containerID="bd89f76b9ca497afc2bc6f33f434a44017705e6cc7bdacb117f0b488c054cbd6" Jul 09 04:40:49.412157 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:49.412086 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd89f76b9ca497afc2bc6f33f434a44017705e6cc7bdacb117f0b488c054cbd6"} err="failed to get container status \"bd89f76b9ca497afc2bc6f33f434a44017705e6cc7bdacb117f0b488c054cbd6\": rpc error: code = NotFound desc = could not find container \"bd89f76b9ca497afc2bc6f33f434a44017705e6cc7bdacb117f0b488c054cbd6\": container with ID starting with bd89f76b9ca497afc2bc6f33f434a44017705e6cc7bdacb117f0b488c054cbd6 not found: ID does not exist" Jul 09 04:40:49.412157 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:49.412106 2565 scope.go:117] "RemoveContainer" containerID="608d9ba7bcbafe28adead8db7033e531241f4268ad3b026a45279b97685fb61d" Jul 09 04:40:49.430627 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:49.430584 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jlmh5" podStartSLOduration=15.430568318 podStartE2EDuration="15.430568318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:40:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:40:49.429139007 +0000 UTC m=+521.680662587" watchObservedRunningTime="2026-07-09 04:40:49.430568318 +0000 UTC m=+521.682091895" Jul 09 04:40:49.442905 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:49.442850 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fw8fm" podStartSLOduration=3.262847821 podStartE2EDuration="25.442833129s" totalImagesPullingTime="22.179985308s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:40:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:40:49.442243207 +0000 UTC m=+521.693766785" watchObservedRunningTime="2026-07-09 04:40:49.442833129 +0000 UTC m=+521.694356707" Jul 09 04:40:49.444185 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:40:49.444159 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vrv4f\" is forbidden: User \"system:node:ip-10-0-139-184.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-184.ec2.internal' and this object" podUID="2cb0aab2-be86-4b85-bb11-ba1f5dacefdb" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vrv4f" Jul 09 04:40:49.445818 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:40:49.445791 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vrv4f\" is forbidden: User \"system:node:ip-10-0-139-184.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-184.ec2.internal' and this object" podUID="2cb0aab2-be86-4b85-bb11-ba1f5dacefdb" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vrv4f" Jul 09 04:40:49.464497 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:49.464467 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c845bc447-d42qw"] Jul 09 04:40:49.467842 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:40:49.467809 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-24vcn\" is forbidden: User \"system:node:ip-10-0-139-184.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-184.ec2.internal' and this object" podUID="807c8d27-3abe-482a-967b-ba3b66a83d17" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-24vcn" Jul 09 04:40:49.468027 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:49.468009 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6c845bc447-d42qw"] Jul 09 04:40:49.761093 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:49.761061 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:40:49.996113 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:49.996085 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:40:50.194920 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:50.194878 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2cb0aab2-be86-4b85-bb11-ba1f5dacefdb" path="/var/lib/kubelet/pods/2cb0aab2-be86-4b85-bb11-ba1f5dacefdb/volumes" Jul 09 04:40:50.195381 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:50.195330 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="680a9c76-039a-4113-bd5e-ff063c22bcdf" path="/var/lib/kubelet/pods/680a9c76-039a-4113-bd5e-ff063c22bcdf/volumes" Jul 09 04:40:50.195686 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:50.195674 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="807c8d27-3abe-482a-967b-ba3b66a83d17" path="/var/lib/kubelet/pods/807c8d27-3abe-482a-967b-ba3b66a83d17/volumes" Jul 09 04:40:51.098009 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:51.097976 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jlmh5"] Jul 09 04:40:51.404858 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:51.404823 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jlmh5" podUID="098d6f62-82ee-4778-a992-83e0f4914889" containerName="manager" containerID="cri-o://929efcc0dbf77cb710b0c085309bfd8f338a5a905bae0e26f19ec3fed643b5cc" gracePeriod=10 Jul 09 04:40:51.651057 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:51.651033 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jlmh5" Jul 09 04:40:51.748520 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:51.748441 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w58tb\" (UniqueName: \"kubernetes.io/projected/098d6f62-82ee-4778-a992-83e0f4914889-kube-api-access-w58tb\") pod \"098d6f62-82ee-4778-a992-83e0f4914889\" (UID: \"098d6f62-82ee-4778-a992-83e0f4914889\") " Jul 09 04:40:51.748520 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:51.748503 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/098d6f62-82ee-4778-a992-83e0f4914889-extensions-socket-volume\") pod \"098d6f62-82ee-4778-a992-83e0f4914889\" (UID: \"098d6f62-82ee-4778-a992-83e0f4914889\") " Jul 09 04:40:51.748868 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:51.748848 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/098d6f62-82ee-4778-a992-83e0f4914889-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "098d6f62-82ee-4778-a992-83e0f4914889" (UID: "098d6f62-82ee-4778-a992-83e0f4914889"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:40:51.750469 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:51.750442 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/098d6f62-82ee-4778-a992-83e0f4914889-kube-api-access-w58tb" (OuterVolumeSpecName: "kube-api-access-w58tb") pod "098d6f62-82ee-4778-a992-83e0f4914889" (UID: "098d6f62-82ee-4778-a992-83e0f4914889"). InnerVolumeSpecName "kube-api-access-w58tb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:40:51.850044 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:51.850009 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w58tb\" (UniqueName: \"kubernetes.io/projected/098d6f62-82ee-4778-a992-83e0f4914889-kube-api-access-w58tb\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:40:51.850044 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:51.850040 2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/098d6f62-82ee-4778-a992-83e0f4914889-extensions-socket-volume\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:40:52.410350 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:52.410303 2565 generic.go:356] "Generic (PLEG): container finished" podID="098d6f62-82ee-4778-a992-83e0f4914889" containerID="929efcc0dbf77cb710b0c085309bfd8f338a5a905bae0e26f19ec3fed643b5cc" exitCode=0 Jul 09 04:40:52.410794 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:52.410361 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jlmh5" event={"ID":"098d6f62-82ee-4778-a992-83e0f4914889","Type":"ContainerDied","Data":"929efcc0dbf77cb710b0c085309bfd8f338a5a905bae0e26f19ec3fed643b5cc"} Jul 09 04:40:52.410794 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:52.410376 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jlmh5" Jul 09 04:40:52.410794 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:52.410404 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jlmh5" event={"ID":"098d6f62-82ee-4778-a992-83e0f4914889","Type":"ContainerDied","Data":"81398295cf79caed37cbadf00b33fa88da5437f7a092c2ff4ba94c29d0de736a"} Jul 09 04:40:52.410794 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:52.410425 2565 scope.go:117] "RemoveContainer" containerID="929efcc0dbf77cb710b0c085309bfd8f338a5a905bae0e26f19ec3fed643b5cc" Jul 09 04:40:52.418866 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:52.418844 2565 scope.go:117] "RemoveContainer" containerID="929efcc0dbf77cb710b0c085309bfd8f338a5a905bae0e26f19ec3fed643b5cc" Jul 09 04:40:52.419124 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:40:52.419107 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"929efcc0dbf77cb710b0c085309bfd8f338a5a905bae0e26f19ec3fed643b5cc\": container with ID starting with 929efcc0dbf77cb710b0c085309bfd8f338a5a905bae0e26f19ec3fed643b5cc not found: ID does not exist" containerID="929efcc0dbf77cb710b0c085309bfd8f338a5a905bae0e26f19ec3fed643b5cc" Jul 09 04:40:52.419176 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:52.419130 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"929efcc0dbf77cb710b0c085309bfd8f338a5a905bae0e26f19ec3fed643b5cc"} err="failed to get container status \"929efcc0dbf77cb710b0c085309bfd8f338a5a905bae0e26f19ec3fed643b5cc\": rpc error: code = NotFound desc = could not find container \"929efcc0dbf77cb710b0c085309bfd8f338a5a905bae0e26f19ec3fed643b5cc\": container with ID starting with 929efcc0dbf77cb710b0c085309bfd8f338a5a905bae0e26f19ec3fed643b5cc not found: ID does not exist" Jul 09 04:40:52.440398 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:52.440360 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jlmh5"] Jul 09 04:40:52.443449 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:52.443419 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jlmh5"] Jul 09 04:40:53.310810 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:53.310776 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:40:53.413149 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:53.413123 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:40:54.195700 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:54.195668 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="098d6f62-82ee-4778-a992-83e0f4914889" path="/var/lib/kubelet/pods/098d6f62-82ee-4778-a992-83e0f4914889/volumes" Jul 09 04:40:59.539235 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:59.539201 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:40:59.934547 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:40:59.934513 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:41:00.035286 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:00.035258 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:41:00.402901 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:00.402821 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zzrz7" Jul 09 04:41:03.211766 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:03.211736 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:41:03.319260 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:03.319229 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:41:07.432718 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.432684 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl"] Jul 09 04:41:07.433186 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.433014 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="098d6f62-82ee-4778-a992-83e0f4914889" containerName="manager" Jul 09 04:41:07.433186 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.433026 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="098d6f62-82ee-4778-a992-83e0f4914889" containerName="manager" Jul 09 04:41:07.433186 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.433035 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="680a9c76-039a-4113-bd5e-ff063c22bcdf" containerName="console" Jul 09 04:41:07.433186 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.433040 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="680a9c76-039a-4113-bd5e-ff063c22bcdf" containerName="console" Jul 09 04:41:07.433186 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.433095 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="680a9c76-039a-4113-bd5e-ff063c22bcdf" containerName="console" Jul 09 04:41:07.433186 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.433104 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="098d6f62-82ee-4778-a992-83e0f4914889" containerName="manager" Jul 09 04:41:07.445719 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.445687 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" Jul 09 04:41:07.447014 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.446980 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl"] Jul 09 04:41:07.448167 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.448148 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 09 04:41:07.448251 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.448162 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-tzqwr\"" Jul 09 04:41:07.459953 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.459886 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:41:07.582088 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.582047 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae4bac90-3e35-4e14-ac3d-126c2f7d274b-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-kwvjl\" (UID: \"ae4bac90-3e35-4e14-ac3d-126c2f7d274b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" Jul 09 04:41:07.582269 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.582120 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae4bac90-3e35-4e14-ac3d-126c2f7d274b-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-kwvjl\" (UID: \"ae4bac90-3e35-4e14-ac3d-126c2f7d274b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" Jul 09 04:41:07.582269 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.582147 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlspn\" (UniqueName: \"kubernetes.io/projected/ae4bac90-3e35-4e14-ac3d-126c2f7d274b-kube-api-access-rlspn\") pod \"maas-default-gateway-openshift-default-56654dcf7-kwvjl\" (UID: \"ae4bac90-3e35-4e14-ac3d-126c2f7d274b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" Jul 09 04:41:07.582269 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.582186 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae4bac90-3e35-4e14-ac3d-126c2f7d274b-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-kwvjl\" (UID: \"ae4bac90-3e35-4e14-ac3d-126c2f7d274b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" Jul 09 04:41:07.582269 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.582239 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae4bac90-3e35-4e14-ac3d-126c2f7d274b-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-kwvjl\" (UID: \"ae4bac90-3e35-4e14-ac3d-126c2f7d274b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" Jul 09 04:41:07.582450 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.582295 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae4bac90-3e35-4e14-ac3d-126c2f7d274b-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-kwvjl\" (UID: \"ae4bac90-3e35-4e14-ac3d-126c2f7d274b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" Jul 09 04:41:07.582450 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.582363 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae4bac90-3e35-4e14-ac3d-126c2f7d274b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-kwvjl\" (UID: \"ae4bac90-3e35-4e14-ac3d-126c2f7d274b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" Jul 09 04:41:07.582450 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.582387 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae4bac90-3e35-4e14-ac3d-126c2f7d274b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-kwvjl\" (UID: \"ae4bac90-3e35-4e14-ac3d-126c2f7d274b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" Jul 09 04:41:07.582450 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.582408 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae4bac90-3e35-4e14-ac3d-126c2f7d274b-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-kwvjl\" (UID: \"ae4bac90-3e35-4e14-ac3d-126c2f7d274b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" Jul 09 04:41:07.683601 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.683521 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae4bac90-3e35-4e14-ac3d-126c2f7d274b-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-kwvjl\" (UID: \"ae4bac90-3e35-4e14-ac3d-126c2f7d274b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" Jul 09 04:41:07.683601 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.683563 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae4bac90-3e35-4e14-ac3d-126c2f7d274b-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-kwvjl\" (UID: \"ae4bac90-3e35-4e14-ac3d-126c2f7d274b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" Jul 09 04:41:07.683601 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.683580 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae4bac90-3e35-4e14-ac3d-126c2f7d274b-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-kwvjl\" (UID: \"ae4bac90-3e35-4e14-ac3d-126c2f7d274b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" Jul 09 04:41:07.683601 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.683604 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae4bac90-3e35-4e14-ac3d-126c2f7d274b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-kwvjl\" (UID: \"ae4bac90-3e35-4e14-ac3d-126c2f7d274b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" Jul 09 04:41:07.683934 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.683623 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae4bac90-3e35-4e14-ac3d-126c2f7d274b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-kwvjl\" (UID: \"ae4bac90-3e35-4e14-ac3d-126c2f7d274b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" Jul 09 04:41:07.683934 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.683644 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae4bac90-3e35-4e14-ac3d-126c2f7d274b-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-kwvjl\" (UID: \"ae4bac90-3e35-4e14-ac3d-126c2f7d274b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" Jul 09 04:41:07.683934 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.683839 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae4bac90-3e35-4e14-ac3d-126c2f7d274b-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-kwvjl\" (UID: \"ae4bac90-3e35-4e14-ac3d-126c2f7d274b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" Jul 09 04:41:07.684092 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.683992 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae4bac90-3e35-4e14-ac3d-126c2f7d274b-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-kwvjl\" (UID: \"ae4bac90-3e35-4e14-ac3d-126c2f7d274b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" Jul 09 04:41:07.684092 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.684010 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae4bac90-3e35-4e14-ac3d-126c2f7d274b-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-kwvjl\" (UID: \"ae4bac90-3e35-4e14-ac3d-126c2f7d274b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" Jul 09 04:41:07.684092 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.684059 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae4bac90-3e35-4e14-ac3d-126c2f7d274b-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-kwvjl\" (UID: \"ae4bac90-3e35-4e14-ac3d-126c2f7d274b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" Jul 09 04:41:07.684228 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.684212 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae4bac90-3e35-4e14-ac3d-126c2f7d274b-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-kwvjl\" (UID: \"ae4bac90-3e35-4e14-ac3d-126c2f7d274b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" Jul 09 04:41:07.684277 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.684242 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlspn\" (UniqueName: \"kubernetes.io/projected/ae4bac90-3e35-4e14-ac3d-126c2f7d274b-kube-api-access-rlspn\") pod \"maas-default-gateway-openshift-default-56654dcf7-kwvjl\" (UID: \"ae4bac90-3e35-4e14-ac3d-126c2f7d274b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" Jul 09 04:41:07.684462 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.684442 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae4bac90-3e35-4e14-ac3d-126c2f7d274b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-kwvjl\" (UID: \"ae4bac90-3e35-4e14-ac3d-126c2f7d274b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" Jul 09 04:41:07.684569 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.684547 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae4bac90-3e35-4e14-ac3d-126c2f7d274b-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-kwvjl\" (UID: \"ae4bac90-3e35-4e14-ac3d-126c2f7d274b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" Jul 09 04:41:07.685926 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.685905 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae4bac90-3e35-4e14-ac3d-126c2f7d274b-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-kwvjl\" (UID: \"ae4bac90-3e35-4e14-ac3d-126c2f7d274b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" Jul 09 04:41:07.686153 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.686134 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae4bac90-3e35-4e14-ac3d-126c2f7d274b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-kwvjl\" (UID: \"ae4bac90-3e35-4e14-ac3d-126c2f7d274b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" Jul 09 04:41:07.691746 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.691723 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlspn\" (UniqueName: \"kubernetes.io/projected/ae4bac90-3e35-4e14-ac3d-126c2f7d274b-kube-api-access-rlspn\") pod \"maas-default-gateway-openshift-default-56654dcf7-kwvjl\" (UID: \"ae4bac90-3e35-4e14-ac3d-126c2f7d274b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" Jul 09 04:41:07.691836 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.691807 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae4bac90-3e35-4e14-ac3d-126c2f7d274b-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-kwvjl\" (UID: \"ae4bac90-3e35-4e14-ac3d-126c2f7d274b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" Jul 09 04:41:07.756890 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.756855 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" Jul 09 04:41:07.918174 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.918152 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl"] Jul 09 04:41:07.920525 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:41:07.920498 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae4bac90_3e35_4e14_ac3d_126c2f7d274b.slice/crio-deda4dfe2ae3236419bcaf48ae614c86fd7a6e674912fb4b840b0bd60fd4febb WatchSource:0}: Error finding container deda4dfe2ae3236419bcaf48ae614c86fd7a6e674912fb4b840b0bd60fd4febb: Status 404 returned error can't find the container with id deda4dfe2ae3236419bcaf48ae614c86fd7a6e674912fb4b840b0bd60fd4febb Jul 09 04:41:07.932850 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:07.932823 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:41:08.490640 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:08.490561 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" event={"ID":"ae4bac90-3e35-4e14-ac3d-126c2f7d274b","Type":"ContainerStarted","Data":"deda4dfe2ae3236419bcaf48ae614c86fd7a6e674912fb4b840b0bd60fd4febb"} Jul 09 04:41:09.332825 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:09.332787 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:41:09.431025 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:09.430987 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:41:09.735675 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:09.735640 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:41:09.787807 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:09.787758 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:41:11.220787 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:11.220748 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:41:11.221084 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:11.220819 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:41:11.221084 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:11.220857 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:41:11.507539 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:11.507442 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" event={"ID":"ae4bac90-3e35-4e14-ac3d-126c2f7d274b","Type":"ContainerStarted","Data":"ef37413ca06efcb6d3b4cec072e0913ac285d281af18b343bc505835829c90dc"} Jul 09 04:41:11.528537 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:11.528475 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" podStartSLOduration=1.230425937 podStartE2EDuration="4.528438927s" totalImagesPullingTime="3.29801299s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:41:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:41:11.52818452 +0000 UTC m=+543.779708097" watchObservedRunningTime="2026-07-09 04:41:11.528438927 +0000 UTC m=+543.779962507" Jul 09 04:41:11.758004 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:11.757925 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" Jul 09 04:41:12.762441 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:12.762409 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" Jul 09 04:41:13.182962 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:13.182914 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:41:13.229574 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:13.229544 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:41:13.363758 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:13.363718 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:41:13.480710 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:13.480632 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:41:13.516919 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:13.516888 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" Jul 09 04:41:13.517908 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:13.517887 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" Jul 09 04:41:21.926359 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:21.926307 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:41:21.936297 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:21.936266 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zbpcx"] Jul 09 04:41:21.964516 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:21.964487 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zbpcx"] Jul 09 04:41:21.964696 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:21.964608 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-zbpcx" Jul 09 04:41:21.967187 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:21.967162 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 09 04:41:22.030867 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:22.030835 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:41:22.039194 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:22.039158 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zbpcx"] Jul 09 04:41:22.109966 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:22.109927 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2998\" (UniqueName: \"kubernetes.io/projected/bacb16b6-87cd-4696-9b59-9cd4b58acca6-kube-api-access-b2998\") pod \"limitador-limitador-64c94fd64f-zbpcx\" (UID: \"bacb16b6-87cd-4696-9b59-9cd4b58acca6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-zbpcx" Jul 09 04:41:22.110141 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:22.110005 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bacb16b6-87cd-4696-9b59-9cd4b58acca6-config-file\") pod \"limitador-limitador-64c94fd64f-zbpcx\" (UID: \"bacb16b6-87cd-4696-9b59-9cd4b58acca6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-zbpcx" Jul 09 04:41:22.177956 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:22.177857 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:41:22.210909 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:22.210875 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bacb16b6-87cd-4696-9b59-9cd4b58acca6-config-file\") pod \"limitador-limitador-64c94fd64f-zbpcx\" (UID: \"bacb16b6-87cd-4696-9b59-9cd4b58acca6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-zbpcx" Jul 09 04:41:22.211048 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:22.210940 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2998\" (UniqueName: \"kubernetes.io/projected/bacb16b6-87cd-4696-9b59-9cd4b58acca6-kube-api-access-b2998\") pod \"limitador-limitador-64c94fd64f-zbpcx\" (UID: \"bacb16b6-87cd-4696-9b59-9cd4b58acca6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-zbpcx" Jul 09 04:41:22.211560 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:22.211542 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bacb16b6-87cd-4696-9b59-9cd4b58acca6-config-file\") pod \"limitador-limitador-64c94fd64f-zbpcx\" (UID: \"bacb16b6-87cd-4696-9b59-9cd4b58acca6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-zbpcx" Jul 09 04:41:22.218866 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:22.218834 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2998\" (UniqueName: \"kubernetes.io/projected/bacb16b6-87cd-4696-9b59-9cd4b58acca6-kube-api-access-b2998\") pod \"limitador-limitador-64c94fd64f-zbpcx\" (UID: \"bacb16b6-87cd-4696-9b59-9cd4b58acca6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-zbpcx" Jul 09 04:41:22.274938 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:22.274902 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-zbpcx" Jul 09 04:41:22.413114 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:22.413087 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zbpcx"] Jul 09 04:41:22.414863 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:41:22.414839 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbacb16b6_87cd_4696_9b59_9cd4b58acca6.slice/crio-0074ba645ec2c3800928a3be19911e1ae008bf0790bf12a20270fe0640a4da2d WatchSource:0}: Error finding container 0074ba645ec2c3800928a3be19911e1ae008bf0790bf12a20270fe0640a4da2d: Status 404 returned error can't find the container with id 0074ba645ec2c3800928a3be19911e1ae008bf0790bf12a20270fe0640a4da2d Jul 09 04:41:22.557042 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:22.556948 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-zbpcx" event={"ID":"bacb16b6-87cd-4696-9b59-9cd4b58acca6","Type":"ContainerStarted","Data":"0074ba645ec2c3800928a3be19911e1ae008bf0790bf12a20270fe0640a4da2d"} Jul 09 04:41:22.703831 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:22.703795 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-xsn4v"] Jul 09 04:41:22.708882 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:22.708857 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xsn4v" Jul 09 04:41:22.711279 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:22.711256 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-lsp8v\"" Jul 09 04:41:22.713609 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:22.713585 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-xsn4v"] Jul 09 04:41:22.738110 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:22.738075 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:41:22.816480 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:22.816390 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pm2g6\" (UniqueName: \"kubernetes.io/projected/8e17c4c3-829d-4b97-80b5-cfce9445aacb-kube-api-access-pm2g6\") pod \"authorino-58cd955675-xsn4v\" (UID: \"8e17c4c3-829d-4b97-80b5-cfce9445aacb\") " pod="kuadrant-system/authorino-58cd955675-xsn4v" Jul 09 04:41:22.822474 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:22.822442 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-72kww"] Jul 09 04:41:22.825844 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:22.825827 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-72kww" Jul 09 04:41:22.832037 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:22.832013 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-72kww"] Jul 09 04:41:22.842607 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:22.842573 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:41:22.917768 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:22.917729 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmcsg\" (UniqueName: \"kubernetes.io/projected/3d49db64-1440-404b-91c9-607b023fb406-kube-api-access-pmcsg\") pod \"authorino-75d556bd77-72kww\" (UID: \"3d49db64-1440-404b-91c9-607b023fb406\") " pod="kuadrant-system/authorino-75d556bd77-72kww" Jul 09 04:41:22.917961 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:22.917794 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pm2g6\" (UniqueName: \"kubernetes.io/projected/8e17c4c3-829d-4b97-80b5-cfce9445aacb-kube-api-access-pm2g6\") pod \"authorino-58cd955675-xsn4v\" (UID: \"8e17c4c3-829d-4b97-80b5-cfce9445aacb\") " pod="kuadrant-system/authorino-58cd955675-xsn4v" Jul 09 04:41:22.926037 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:22.926012 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pm2g6\" (UniqueName: \"kubernetes.io/projected/8e17c4c3-829d-4b97-80b5-cfce9445aacb-kube-api-access-pm2g6\") pod \"authorino-58cd955675-xsn4v\" (UID: \"8e17c4c3-829d-4b97-80b5-cfce9445aacb\") " pod="kuadrant-system/authorino-58cd955675-xsn4v" Jul 09 04:41:23.018731 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:23.018697 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmcsg\" (UniqueName: \"kubernetes.io/projected/3d49db64-1440-404b-91c9-607b023fb406-kube-api-access-pmcsg\") pod \"authorino-75d556bd77-72kww\" (UID: \"3d49db64-1440-404b-91c9-607b023fb406\") " pod="kuadrant-system/authorino-75d556bd77-72kww" Jul 09 04:41:23.019111 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:23.018961 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xsn4v" Jul 09 04:41:23.026045 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:23.026023 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmcsg\" (UniqueName: \"kubernetes.io/projected/3d49db64-1440-404b-91c9-607b023fb406-kube-api-access-pmcsg\") pod \"authorino-75d556bd77-72kww\" (UID: \"3d49db64-1440-404b-91c9-607b023fb406\") " pod="kuadrant-system/authorino-75d556bd77-72kww" Jul 09 04:41:23.136075 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:23.136047 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-72kww" Jul 09 04:41:23.149025 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:23.149002 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-xsn4v"] Jul 09 04:41:23.151088 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:41:23.151062 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8e17c4c3_829d_4b97_80b5_cfce9445aacb.slice/crio-6e062dde8e2f8e628cd7edd2f061b537d31ca6e2dab5b3d7ffbf41605c0e029c WatchSource:0}: Error finding container 6e062dde8e2f8e628cd7edd2f061b537d31ca6e2dab5b3d7ffbf41605c0e029c: Status 404 returned error can't find the container with id 6e062dde8e2f8e628cd7edd2f061b537d31ca6e2dab5b3d7ffbf41605c0e029c Jul 09 04:41:23.287255 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:23.286739 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-72kww"] Jul 09 04:41:23.565721 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:23.565685 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-72kww" event={"ID":"3d49db64-1440-404b-91c9-607b023fb406","Type":"ContainerStarted","Data":"65c0fdcedecea7a40d102ab2f6e96351745362247648ceb932ce7a3a1440e57a"} Jul 09 04:41:23.567086 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:23.567055 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xsn4v" event={"ID":"8e17c4c3-829d-4b97-80b5-cfce9445aacb","Type":"ContainerStarted","Data":"6e062dde8e2f8e628cd7edd2f061b537d31ca6e2dab5b3d7ffbf41605c0e029c"} Jul 09 04:41:28.609907 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:28.609870 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-zbpcx" event={"ID":"bacb16b6-87cd-4696-9b59-9cd4b58acca6","Type":"ContainerStarted","Data":"1dac468e6abf340e216cdf59759defdd5fdee40ac86dd6ae04ca02a60c5c4fca"} Jul 09 04:41:28.610308 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:28.609989 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-zbpcx" Jul 09 04:41:28.611469 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:28.611444 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xsn4v" event={"ID":"8e17c4c3-829d-4b97-80b5-cfce9445aacb","Type":"ContainerStarted","Data":"530f5e95a9125cba104c0b9b03dae0e3ae70145eaa884680aff60cbca4369325"} Jul 09 04:41:28.612803 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:28.612779 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-72kww" event={"ID":"3d49db64-1440-404b-91c9-607b023fb406","Type":"ContainerStarted","Data":"1a26966701c4f92773d498426164ecdd894d4ab761dc17e8eda98cfee2a49200"} Jul 09 04:41:28.625240 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:28.625183 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-zbpcx" podStartSLOduration=2.327032959 podStartE2EDuration="7.625163751s" totalImagesPullingTime="5.298130792s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:41:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:41:28.624702109 +0000 UTC m=+560.876225687" watchObservedRunningTime="2026-07-09 04:41:28.625163751 +0000 UTC m=+560.876687356" Jul 09 04:41:28.639740 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:28.639690 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-72kww" podStartSLOduration=2.224428349 podStartE2EDuration="6.639674705s" totalImagesPullingTime="4.415246356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:41:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:41:28.638863535 +0000 UTC m=+560.890387112" watchObservedRunningTime="2026-07-09 04:41:28.639674705 +0000 UTC m=+560.891198283" Jul 09 04:41:28.653862 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:28.653811 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-xsn4v" podStartSLOduration=2.098193823 podStartE2EDuration="6.653795489s" totalImagesPullingTime="4.555601666s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:41:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:41:28.652527622 +0000 UTC m=+560.904051191" watchObservedRunningTime="2026-07-09 04:41:28.653795489 +0000 UTC m=+560.905319067" Jul 09 04:41:28.705069 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:28.705036 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-xsn4v"] Jul 09 04:41:30.623420 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:30.623371 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-xsn4v" podUID="8e17c4c3-829d-4b97-80b5-cfce9445aacb" containerName="authorino" containerID="cri-o://530f5e95a9125cba104c0b9b03dae0e3ae70145eaa884680aff60cbca4369325" gracePeriod=30 Jul 09 04:41:30.862229 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:30.862206 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xsn4v" Jul 09 04:41:30.994234 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:30.994199 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pm2g6\" (UniqueName: \"kubernetes.io/projected/8e17c4c3-829d-4b97-80b5-cfce9445aacb-kube-api-access-pm2g6\") pod \"8e17c4c3-829d-4b97-80b5-cfce9445aacb\" (UID: \"8e17c4c3-829d-4b97-80b5-cfce9445aacb\") " Jul 09 04:41:30.996274 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:30.996248 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e17c4c3-829d-4b97-80b5-cfce9445aacb-kube-api-access-pm2g6" (OuterVolumeSpecName: "kube-api-access-pm2g6") pod "8e17c4c3-829d-4b97-80b5-cfce9445aacb" (UID: "8e17c4c3-829d-4b97-80b5-cfce9445aacb"). InnerVolumeSpecName "kube-api-access-pm2g6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:41:31.095061 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:31.095027 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pm2g6\" (UniqueName: \"kubernetes.io/projected/8e17c4c3-829d-4b97-80b5-cfce9445aacb-kube-api-access-pm2g6\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:41:31.630594 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:31.630559 2565 generic.go:356] "Generic (PLEG): container finished" podID="8e17c4c3-829d-4b97-80b5-cfce9445aacb" containerID="530f5e95a9125cba104c0b9b03dae0e3ae70145eaa884680aff60cbca4369325" exitCode=0 Jul 09 04:41:31.630594 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:31.630597 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xsn4v" Jul 09 04:41:31.631148 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:31.630648 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xsn4v" event={"ID":"8e17c4c3-829d-4b97-80b5-cfce9445aacb","Type":"ContainerDied","Data":"530f5e95a9125cba104c0b9b03dae0e3ae70145eaa884680aff60cbca4369325"} Jul 09 04:41:31.631148 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:31.630693 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xsn4v" event={"ID":"8e17c4c3-829d-4b97-80b5-cfce9445aacb","Type":"ContainerDied","Data":"6e062dde8e2f8e628cd7edd2f061b537d31ca6e2dab5b3d7ffbf41605c0e029c"} Jul 09 04:41:31.631148 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:31.630716 2565 scope.go:117] "RemoveContainer" containerID="530f5e95a9125cba104c0b9b03dae0e3ae70145eaa884680aff60cbca4369325" Jul 09 04:41:31.639724 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:31.639709 2565 scope.go:117] "RemoveContainer" containerID="530f5e95a9125cba104c0b9b03dae0e3ae70145eaa884680aff60cbca4369325" Jul 09 04:41:31.639998 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:41:31.639977 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"530f5e95a9125cba104c0b9b03dae0e3ae70145eaa884680aff60cbca4369325\": container with ID starting with 530f5e95a9125cba104c0b9b03dae0e3ae70145eaa884680aff60cbca4369325 not found: ID does not exist" containerID="530f5e95a9125cba104c0b9b03dae0e3ae70145eaa884680aff60cbca4369325" Jul 09 04:41:31.640045 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:31.640007 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"530f5e95a9125cba104c0b9b03dae0e3ae70145eaa884680aff60cbca4369325"} err="failed to get container status \"530f5e95a9125cba104c0b9b03dae0e3ae70145eaa884680aff60cbca4369325\": rpc error: code = NotFound desc = could not find container \"530f5e95a9125cba104c0b9b03dae0e3ae70145eaa884680aff60cbca4369325\": container with ID starting with 530f5e95a9125cba104c0b9b03dae0e3ae70145eaa884680aff60cbca4369325 not found: ID does not exist" Jul 09 04:41:31.653480 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:31.653457 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-xsn4v"] Jul 09 04:41:31.657142 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:31.657117 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-xsn4v"] Jul 09 04:41:32.195819 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:32.195789 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e17c4c3-829d-4b97-80b5-cfce9445aacb" path="/var/lib/kubelet/pods/8e17c4c3-829d-4b97-80b5-cfce9445aacb/volumes" Jul 09 04:41:36.963288 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:36.963250 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zbpcx"] Jul 09 04:41:36.963771 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:36.963481 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-zbpcx" podUID="bacb16b6-87cd-4696-9b59-9cd4b58acca6" containerName="limitador" containerID="cri-o://1dac468e6abf340e216cdf59759defdd5fdee40ac86dd6ae04ca02a60c5c4fca" gracePeriod=30 Jul 09 04:41:36.964155 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:36.964134 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-zbpcx" Jul 09 04:41:37.659457 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:37.659426 2565 generic.go:356] "Generic (PLEG): container finished" podID="bacb16b6-87cd-4696-9b59-9cd4b58acca6" containerID="1dac468e6abf340e216cdf59759defdd5fdee40ac86dd6ae04ca02a60c5c4fca" exitCode=0 Jul 09 04:41:37.659625 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:37.659500 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-zbpcx" event={"ID":"bacb16b6-87cd-4696-9b59-9cd4b58acca6","Type":"ContainerDied","Data":"1dac468e6abf340e216cdf59759defdd5fdee40ac86dd6ae04ca02a60c5c4fca"} Jul 09 04:41:37.903571 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:37.903548 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-zbpcx" Jul 09 04:41:37.956707 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:37.956612 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bacb16b6-87cd-4696-9b59-9cd4b58acca6-config-file\") pod \"bacb16b6-87cd-4696-9b59-9cd4b58acca6\" (UID: \"bacb16b6-87cd-4696-9b59-9cd4b58acca6\") " Jul 09 04:41:37.956707 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:37.956667 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2998\" (UniqueName: \"kubernetes.io/projected/bacb16b6-87cd-4696-9b59-9cd4b58acca6-kube-api-access-b2998\") pod \"bacb16b6-87cd-4696-9b59-9cd4b58acca6\" (UID: \"bacb16b6-87cd-4696-9b59-9cd4b58acca6\") " Jul 09 04:41:37.956980 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:37.956956 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bacb16b6-87cd-4696-9b59-9cd4b58acca6-config-file" (OuterVolumeSpecName: "config-file") pod "bacb16b6-87cd-4696-9b59-9cd4b58acca6" (UID: "bacb16b6-87cd-4696-9b59-9cd4b58acca6"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:41:37.958764 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:37.958741 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bacb16b6-87cd-4696-9b59-9cd4b58acca6-kube-api-access-b2998" (OuterVolumeSpecName: "kube-api-access-b2998") pod "bacb16b6-87cd-4696-9b59-9cd4b58acca6" (UID: "bacb16b6-87cd-4696-9b59-9cd4b58acca6"). InnerVolumeSpecName "kube-api-access-b2998". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:41:38.057901 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:38.057850 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2998\" (UniqueName: \"kubernetes.io/projected/bacb16b6-87cd-4696-9b59-9cd4b58acca6-kube-api-access-b2998\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:41:38.057901 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:38.057893 2565 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bacb16b6-87cd-4696-9b59-9cd4b58acca6-config-file\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:41:38.664877 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:38.664840 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-zbpcx" Jul 09 04:41:38.664877 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:38.664859 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-zbpcx" event={"ID":"bacb16b6-87cd-4696-9b59-9cd4b58acca6","Type":"ContainerDied","Data":"0074ba645ec2c3800928a3be19911e1ae008bf0790bf12a20270fe0640a4da2d"} Jul 09 04:41:38.665144 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:38.664927 2565 scope.go:117] "RemoveContainer" containerID="1dac468e6abf340e216cdf59759defdd5fdee40ac86dd6ae04ca02a60c5c4fca" Jul 09 04:41:38.680027 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:38.680003 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zbpcx"] Jul 09 04:41:38.684323 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:38.684305 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zbpcx"] Jul 09 04:41:38.804699 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:38.804668 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:41:38.903701 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:38.903668 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:41:39.227191 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:39.227162 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-rt2kf"] Jul 09 04:41:39.227603 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:39.227529 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e17c4c3-829d-4b97-80b5-cfce9445aacb" containerName="authorino" Jul 09 04:41:39.227603 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:39.227542 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e17c4c3-829d-4b97-80b5-cfce9445aacb" containerName="authorino" Jul 09 04:41:39.227603 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:39.227553 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bacb16b6-87cd-4696-9b59-9cd4b58acca6" containerName="limitador" Jul 09 04:41:39.227603 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:39.227558 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="bacb16b6-87cd-4696-9b59-9cd4b58acca6" containerName="limitador" Jul 09 04:41:39.227749 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:39.227609 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="bacb16b6-87cd-4696-9b59-9cd4b58acca6" containerName="limitador" Jul 09 04:41:39.227749 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:39.227619 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e17c4c3-829d-4b97-80b5-cfce9445aacb" containerName="authorino" Jul 09 04:41:39.231741 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:39.231723 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rt2kf" Jul 09 04:41:39.234368 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:39.234349 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 09 04:41:39.234486 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:39.234386 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-ls88s\"" Jul 09 04:41:39.234486 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:39.234332 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 09 04:41:39.234486 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:39.234466 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 09 04:41:39.239374 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:39.239111 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-rt2kf"] Jul 09 04:41:39.265405 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:39.265376 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:41:39.267477 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:39.267450 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e7b058ee-0499-4c8b-800d-0278de0eddec-data\") pod \"postgres-69fb4594cf-rt2kf\" (UID: \"e7b058ee-0499-4c8b-800d-0278de0eddec\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rt2kf" Jul 09 04:41:39.267783 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:39.267765 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxknq\" (UniqueName: \"kubernetes.io/projected/e7b058ee-0499-4c8b-800d-0278de0eddec-kube-api-access-sxknq\") pod \"postgres-69fb4594cf-rt2kf\" (UID: \"e7b058ee-0499-4c8b-800d-0278de0eddec\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rt2kf" Jul 09 04:41:39.368898 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:39.368865 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e7b058ee-0499-4c8b-800d-0278de0eddec-data\") pod \"postgres-69fb4594cf-rt2kf\" (UID: \"e7b058ee-0499-4c8b-800d-0278de0eddec\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rt2kf" Jul 09 04:41:39.369087 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:39.368933 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxknq\" (UniqueName: \"kubernetes.io/projected/e7b058ee-0499-4c8b-800d-0278de0eddec-kube-api-access-sxknq\") pod \"postgres-69fb4594cf-rt2kf\" (UID: \"e7b058ee-0499-4c8b-800d-0278de0eddec\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rt2kf" Jul 09 04:41:39.369326 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:39.369305 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e7b058ee-0499-4c8b-800d-0278de0eddec-data\") pod \"postgres-69fb4594cf-rt2kf\" (UID: \"e7b058ee-0499-4c8b-800d-0278de0eddec\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rt2kf" Jul 09 04:41:39.377785 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:39.377762 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxknq\" (UniqueName: \"kubernetes.io/projected/e7b058ee-0499-4c8b-800d-0278de0eddec-kube-api-access-sxknq\") pod \"postgres-69fb4594cf-rt2kf\" (UID: \"e7b058ee-0499-4c8b-800d-0278de0eddec\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rt2kf" Jul 09 04:41:39.543721 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:39.543645 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rt2kf" Jul 09 04:41:39.672285 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:39.672255 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-rt2kf"] Jul 09 04:41:39.676778 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:41:39.676753 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7b058ee_0499_4c8b_800d_0278de0eddec.slice/crio-03b746f453d36d5739643f4b4a99bcf278000139d2ba717db61f900999679f98 WatchSource:0}: Error finding container 03b746f453d36d5739643f4b4a99bcf278000139d2ba717db61f900999679f98: Status 404 returned error can't find the container with id 03b746f453d36d5739643f4b4a99bcf278000139d2ba717db61f900999679f98 Jul 09 04:41:40.195185 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:40.195157 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bacb16b6-87cd-4696-9b59-9cd4b58acca6" path="/var/lib/kubelet/pods/bacb16b6-87cd-4696-9b59-9cd4b58acca6/volumes" Jul 09 04:41:40.680519 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:40.680480 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rt2kf" event={"ID":"e7b058ee-0499-4c8b-800d-0278de0eddec","Type":"ContainerStarted","Data":"03b746f453d36d5739643f4b4a99bcf278000139d2ba717db61f900999679f98"} Jul 09 04:41:44.703560 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:44.703520 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rt2kf" event={"ID":"e7b058ee-0499-4c8b-800d-0278de0eddec","Type":"ContainerStarted","Data":"5a6a4689806903b10ece068654d3dfe41d692707640d6af98cea4660e8b13aeb"} Jul 09 04:41:44.703942 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:44.703635 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rt2kf" Jul 09 04:41:44.719079 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:44.718986 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rt2kf" podStartSLOduration=0.92744575 podStartE2EDuration="5.718970525s" totalImagesPullingTime="4.791524775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:41:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:41:44.717647656 +0000 UTC m=+576.969171235" watchObservedRunningTime="2026-07-09 04:41:44.718970525 +0000 UTC m=+576.970494103" Jul 09 04:41:50.739474 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:50.739442 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rt2kf" Jul 09 04:41:51.043427 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:51.039729 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:41:51.265526 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:51.265494 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-5kk9h"] Jul 09 04:41:51.269184 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:51.269162 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-5kk9h" Jul 09 04:41:51.276097 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:51.276057 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-5kk9h"] Jul 09 04:41:51.316135 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:51.316036 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:41:51.375485 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:51.375448 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4z8nb\" (UniqueName: \"kubernetes.io/projected/66ca5e10-c082-4577-a83a-0618943eeada-kube-api-access-4z8nb\") pod \"authorino-5d4b67c446-5kk9h\" (UID: \"66ca5e10-c082-4577-a83a-0618943eeada\") " pod="kuadrant-system/authorino-5d4b67c446-5kk9h" Jul 09 04:41:51.476649 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:51.476618 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z8nb\" (UniqueName: \"kubernetes.io/projected/66ca5e10-c082-4577-a83a-0618943eeada-kube-api-access-4z8nb\") pod \"authorino-5d4b67c446-5kk9h\" (UID: \"66ca5e10-c082-4577-a83a-0618943eeada\") " pod="kuadrant-system/authorino-5d4b67c446-5kk9h" Jul 09 04:41:51.485385 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:51.485356 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z8nb\" (UniqueName: \"kubernetes.io/projected/66ca5e10-c082-4577-a83a-0618943eeada-kube-api-access-4z8nb\") pod \"authorino-5d4b67c446-5kk9h\" (UID: \"66ca5e10-c082-4577-a83a-0618943eeada\") " pod="kuadrant-system/authorino-5d4b67c446-5kk9h" Jul 09 04:41:51.510597 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:51.510565 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-5kk9h"] Jul 09 04:41:51.510871 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:51.510855 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-5kk9h" Jul 09 04:41:51.533998 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:51.533971 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5dc45bbd98-w74mh"] Jul 09 04:41:51.542533 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:51.542508 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc45bbd98-w74mh" Jul 09 04:41:51.543636 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:51.543608 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5dc45bbd98-w74mh"] Jul 09 04:41:51.579930 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:51.579852 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc45bbd98-w74mh"] Jul 09 04:41:51.580210 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:41:51.580177 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-bcxqr], unattached volumes=[], failed to process volumes=[kube-api-access-bcxqr]: context canceled" pod="kuadrant-system/authorino-5dc45bbd98-w74mh" podUID="215eb8cd-69b0-4731-ad12-3d5d87a509e5" Jul 09 04:41:51.609019 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:51.608988 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5fbd8d9b6-58hdv"] Jul 09 04:41:51.612871 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:51.612850 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5fbd8d9b6-58hdv" Jul 09 04:41:51.615595 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:51.615358 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 09 04:41:51.617734 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:51.617684 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5fbd8d9b6-58hdv"] Jul 09 04:41:51.647659 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:51.647633 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-5kk9h"] Jul 09 04:41:51.651242 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:41:51.651211 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod66ca5e10_c082_4577_a83a_0618943eeada.slice/crio-ee2c266db7ad524edbed6a21fa1c1c8cf2a21e632380d2d0210107275959982d WatchSource:0}: Error finding container ee2c266db7ad524edbed6a21fa1c1c8cf2a21e632380d2d0210107275959982d: Status 404 returned error can't find the container with id ee2c266db7ad524edbed6a21fa1c1c8cf2a21e632380d2d0210107275959982d Jul 09 04:41:51.678580 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:51.678544 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcxqr\" (UniqueName: \"kubernetes.io/projected/215eb8cd-69b0-4731-ad12-3d5d87a509e5-kube-api-access-bcxqr\") pod \"authorino-5dc45bbd98-w74mh\" (UID: \"215eb8cd-69b0-4731-ad12-3d5d87a509e5\") " pod="kuadrant-system/authorino-5dc45bbd98-w74mh" Jul 09 04:41:51.678745 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:51.678604 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rz62\" (UniqueName: \"kubernetes.io/projected/dd0e0e40-a52a-449d-b358-cabef23533e2-kube-api-access-9rz62\") pod \"authorino-5fbd8d9b6-58hdv\" (UID: \"dd0e0e40-a52a-449d-b358-cabef23533e2\") " pod="kuadrant-system/authorino-5fbd8d9b6-58hdv" Jul 09 04:41:51.678745 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:51.678645 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dd0e0e40-a52a-449d-b358-cabef23533e2-tls-cert\") pod \"authorino-5fbd8d9b6-58hdv\" (UID: \"dd0e0e40-a52a-449d-b358-cabef23533e2\") " pod="kuadrant-system/authorino-5fbd8d9b6-58hdv" Jul 09 04:41:51.737177 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:51.737145 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-5kk9h" event={"ID":"66ca5e10-c082-4577-a83a-0618943eeada","Type":"ContainerStarted","Data":"ee2c266db7ad524edbed6a21fa1c1c8cf2a21e632380d2d0210107275959982d"} Jul 09 04:41:51.737380 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:51.737193 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc45bbd98-w74mh" Jul 09 04:41:51.744167 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:51.744146 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc45bbd98-w74mh" Jul 09 04:41:51.779172 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:51.779141 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rz62\" (UniqueName: \"kubernetes.io/projected/dd0e0e40-a52a-449d-b358-cabef23533e2-kube-api-access-9rz62\") pod \"authorino-5fbd8d9b6-58hdv\" (UID: \"dd0e0e40-a52a-449d-b358-cabef23533e2\") " pod="kuadrant-system/authorino-5fbd8d9b6-58hdv" Jul 09 04:41:51.779331 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:51.779184 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dd0e0e40-a52a-449d-b358-cabef23533e2-tls-cert\") pod \"authorino-5fbd8d9b6-58hdv\" (UID: \"dd0e0e40-a52a-449d-b358-cabef23533e2\") " pod="kuadrant-system/authorino-5fbd8d9b6-58hdv" Jul 09 04:41:51.779331 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:51.779257 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcxqr\" (UniqueName: \"kubernetes.io/projected/215eb8cd-69b0-4731-ad12-3d5d87a509e5-kube-api-access-bcxqr\") pod \"authorino-5dc45bbd98-w74mh\" (UID: \"215eb8cd-69b0-4731-ad12-3d5d87a509e5\") " pod="kuadrant-system/authorino-5dc45bbd98-w74mh" Jul 09 04:41:51.781593 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:51.781574 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dd0e0e40-a52a-449d-b358-cabef23533e2-tls-cert\") pod \"authorino-5fbd8d9b6-58hdv\" (UID: \"dd0e0e40-a52a-449d-b358-cabef23533e2\") " pod="kuadrant-system/authorino-5fbd8d9b6-58hdv" Jul 09 04:41:51.786772 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:51.786747 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcxqr\" (UniqueName: \"kubernetes.io/projected/215eb8cd-69b0-4731-ad12-3d5d87a509e5-kube-api-access-bcxqr\") pod \"authorino-5dc45bbd98-w74mh\" (UID: \"215eb8cd-69b0-4731-ad12-3d5d87a509e5\") " pod="kuadrant-system/authorino-5dc45bbd98-w74mh" Jul 09 04:41:51.787177 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:51.787159 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rz62\" (UniqueName: \"kubernetes.io/projected/dd0e0e40-a52a-449d-b358-cabef23533e2-kube-api-access-9rz62\") pod \"authorino-5fbd8d9b6-58hdv\" (UID: \"dd0e0e40-a52a-449d-b358-cabef23533e2\") " pod="kuadrant-system/authorino-5fbd8d9b6-58hdv" Jul 09 04:41:51.880412 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:51.880321 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bcxqr\" (UniqueName: \"kubernetes.io/projected/215eb8cd-69b0-4731-ad12-3d5d87a509e5-kube-api-access-bcxqr\") pod \"215eb8cd-69b0-4731-ad12-3d5d87a509e5\" (UID: \"215eb8cd-69b0-4731-ad12-3d5d87a509e5\") " Jul 09 04:41:51.882293 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:51.882270 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/215eb8cd-69b0-4731-ad12-3d5d87a509e5-kube-api-access-bcxqr" (OuterVolumeSpecName: "kube-api-access-bcxqr") pod "215eb8cd-69b0-4731-ad12-3d5d87a509e5" (UID: "215eb8cd-69b0-4731-ad12-3d5d87a509e5"). InnerVolumeSpecName "kube-api-access-bcxqr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:41:51.924138 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:51.924108 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5fbd8d9b6-58hdv" Jul 09 04:41:51.981079 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:51.981054 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bcxqr\" (UniqueName: \"kubernetes.io/projected/215eb8cd-69b0-4731-ad12-3d5d87a509e5-kube-api-access-bcxqr\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:41:52.058534 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:52.058509 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5fbd8d9b6-58hdv"] Jul 09 04:41:52.059824 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:41:52.059797 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddd0e0e40_a52a_449d_b358_cabef23533e2.slice/crio-f7226038ee208057150f5e54161292a2989b6a98a8c1eeebb3024a1d605284c9 WatchSource:0}: Error finding container f7226038ee208057150f5e54161292a2989b6a98a8c1eeebb3024a1d605284c9: Status 404 returned error can't find the container with id f7226038ee208057150f5e54161292a2989b6a98a8c1eeebb3024a1d605284c9 Jul 09 04:41:52.743584 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:52.743544 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5fbd8d9b6-58hdv" event={"ID":"dd0e0e40-a52a-449d-b358-cabef23533e2","Type":"ContainerStarted","Data":"ebcc204a17da45eb4db2ee9c23d10988b3463c39d243d87fcd66e62182f8a07b"} Jul 09 04:41:52.743779 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:52.743595 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5fbd8d9b6-58hdv" event={"ID":"dd0e0e40-a52a-449d-b358-cabef23533e2","Type":"ContainerStarted","Data":"f7226038ee208057150f5e54161292a2989b6a98a8c1eeebb3024a1d605284c9"} Jul 09 04:41:52.745126 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:52.745097 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-5kk9h" event={"ID":"66ca5e10-c082-4577-a83a-0618943eeada","Type":"ContainerStarted","Data":"98d9d7ad1c2c055b06712f286bdfe219bffd4a4f9de38a09bd6657be584cd265"} Jul 09 04:41:52.745126 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:52.745112 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-5kk9h" podUID="66ca5e10-c082-4577-a83a-0618943eeada" containerName="authorino" containerID="cri-o://98d9d7ad1c2c055b06712f286bdfe219bffd4a4f9de38a09bd6657be584cd265" gracePeriod=30 Jul 09 04:41:52.745552 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:52.745104 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc45bbd98-w74mh" Jul 09 04:41:52.758700 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:52.758661 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5fbd8d9b6-58hdv" podStartSLOduration=1.2831460780000001 podStartE2EDuration="1.758644505s" totalImagesPullingTime="475.498427ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:41:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:41:52.756795744 +0000 UTC m=+585.008319324" watchObservedRunningTime="2026-07-09 04:41:52.758644505 +0000 UTC m=+585.010168083" Jul 09 04:41:52.770939 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:52.770889 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-5kk9h" podStartSLOduration=1.422137544 podStartE2EDuration="1.770869843s" totalImagesPullingTime="348.732299ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:41:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:41:52.769834674 +0000 UTC m=+585.021358250" watchObservedRunningTime="2026-07-09 04:41:52.770869843 +0000 UTC m=+585.022393421" Jul 09 04:41:52.781375 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:52.781329 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-72kww"] Jul 09 04:41:52.781554 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:52.781534 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-72kww" podUID="3d49db64-1440-404b-91c9-607b023fb406" containerName="authorino" containerID="cri-o://1a26966701c4f92773d498426164ecdd894d4ab761dc17e8eda98cfee2a49200" gracePeriod=30 Jul 09 04:41:52.794670 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:52.794589 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc45bbd98-w74mh"] Jul 09 04:41:52.797308 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:52.797276 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5dc45bbd98-w74mh"] Jul 09 04:41:53.034570 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:53.034546 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-5kk9h" Jul 09 04:41:53.042372 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:53.042334 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-72kww" Jul 09 04:41:53.092735 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:53.092652 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmcsg\" (UniqueName: \"kubernetes.io/projected/3d49db64-1440-404b-91c9-607b023fb406-kube-api-access-pmcsg\") pod \"3d49db64-1440-404b-91c9-607b023fb406\" (UID: \"3d49db64-1440-404b-91c9-607b023fb406\") " Jul 09 04:41:53.092735 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:53.092685 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4z8nb\" (UniqueName: \"kubernetes.io/projected/66ca5e10-c082-4577-a83a-0618943eeada-kube-api-access-4z8nb\") pod \"66ca5e10-c082-4577-a83a-0618943eeada\" (UID: \"66ca5e10-c082-4577-a83a-0618943eeada\") " Jul 09 04:41:53.094718 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:53.094690 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d49db64-1440-404b-91c9-607b023fb406-kube-api-access-pmcsg" (OuterVolumeSpecName: "kube-api-access-pmcsg") pod "3d49db64-1440-404b-91c9-607b023fb406" (UID: "3d49db64-1440-404b-91c9-607b023fb406"). InnerVolumeSpecName "kube-api-access-pmcsg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:41:53.094718 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:53.094697 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66ca5e10-c082-4577-a83a-0618943eeada-kube-api-access-4z8nb" (OuterVolumeSpecName: "kube-api-access-4z8nb") pod "66ca5e10-c082-4577-a83a-0618943eeada" (UID: "66ca5e10-c082-4577-a83a-0618943eeada"). InnerVolumeSpecName "kube-api-access-4z8nb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:41:53.194224 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:53.194187 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmcsg\" (UniqueName: \"kubernetes.io/projected/3d49db64-1440-404b-91c9-607b023fb406-kube-api-access-pmcsg\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:41:53.194224 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:53.194215 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4z8nb\" (UniqueName: \"kubernetes.io/projected/66ca5e10-c082-4577-a83a-0618943eeada-kube-api-access-4z8nb\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:41:53.751127 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:53.751033 2565 generic.go:356] "Generic (PLEG): container finished" podID="66ca5e10-c082-4577-a83a-0618943eeada" containerID="98d9d7ad1c2c055b06712f286bdfe219bffd4a4f9de38a09bd6657be584cd265" exitCode=0 Jul 09 04:41:53.751127 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:53.751077 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-5kk9h" Jul 09 04:41:53.751679 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:53.751124 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-5kk9h" event={"ID":"66ca5e10-c082-4577-a83a-0618943eeada","Type":"ContainerDied","Data":"98d9d7ad1c2c055b06712f286bdfe219bffd4a4f9de38a09bd6657be584cd265"} Jul 09 04:41:53.751679 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:53.751161 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-5kk9h" event={"ID":"66ca5e10-c082-4577-a83a-0618943eeada","Type":"ContainerDied","Data":"ee2c266db7ad524edbed6a21fa1c1c8cf2a21e632380d2d0210107275959982d"} Jul 09 04:41:53.751679 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:53.751178 2565 scope.go:117] "RemoveContainer" containerID="98d9d7ad1c2c055b06712f286bdfe219bffd4a4f9de38a09bd6657be584cd265" Jul 09 04:41:53.752472 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:53.752373 2565 generic.go:356] "Generic (PLEG): container finished" podID="3d49db64-1440-404b-91c9-607b023fb406" containerID="1a26966701c4f92773d498426164ecdd894d4ab761dc17e8eda98cfee2a49200" exitCode=0 Jul 09 04:41:53.752472 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:53.752403 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-72kww" event={"ID":"3d49db64-1440-404b-91c9-607b023fb406","Type":"ContainerDied","Data":"1a26966701c4f92773d498426164ecdd894d4ab761dc17e8eda98cfee2a49200"} Jul 09 04:41:53.752472 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:53.752445 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-72kww" event={"ID":"3d49db64-1440-404b-91c9-607b023fb406","Type":"ContainerDied","Data":"65c0fdcedecea7a40d102ab2f6e96351745362247648ceb932ce7a3a1440e57a"} Jul 09 04:41:53.752472 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:53.752413 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-72kww" Jul 09 04:41:53.764316 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:53.760880 2565 scope.go:117] "RemoveContainer" containerID="98d9d7ad1c2c055b06712f286bdfe219bffd4a4f9de38a09bd6657be584cd265" Jul 09 04:41:53.764316 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:41:53.761520 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98d9d7ad1c2c055b06712f286bdfe219bffd4a4f9de38a09bd6657be584cd265\": container with ID starting with 98d9d7ad1c2c055b06712f286bdfe219bffd4a4f9de38a09bd6657be584cd265 not found: ID does not exist" containerID="98d9d7ad1c2c055b06712f286bdfe219bffd4a4f9de38a09bd6657be584cd265" Jul 09 04:41:53.764316 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:53.761672 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98d9d7ad1c2c055b06712f286bdfe219bffd4a4f9de38a09bd6657be584cd265"} err="failed to get container status \"98d9d7ad1c2c055b06712f286bdfe219bffd4a4f9de38a09bd6657be584cd265\": rpc error: code = NotFound desc = could not find container \"98d9d7ad1c2c055b06712f286bdfe219bffd4a4f9de38a09bd6657be584cd265\": container with ID starting with 98d9d7ad1c2c055b06712f286bdfe219bffd4a4f9de38a09bd6657be584cd265 not found: ID does not exist" Jul 09 04:41:53.764316 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:53.761745 2565 scope.go:117] "RemoveContainer" containerID="1a26966701c4f92773d498426164ecdd894d4ab761dc17e8eda98cfee2a49200" Jul 09 04:41:53.772021 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:53.772002 2565 scope.go:117] "RemoveContainer" containerID="1a26966701c4f92773d498426164ecdd894d4ab761dc17e8eda98cfee2a49200" Jul 09 04:41:53.772269 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:41:53.772250 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a26966701c4f92773d498426164ecdd894d4ab761dc17e8eda98cfee2a49200\": container with ID starting with 1a26966701c4f92773d498426164ecdd894d4ab761dc17e8eda98cfee2a49200 not found: ID does not exist" containerID="1a26966701c4f92773d498426164ecdd894d4ab761dc17e8eda98cfee2a49200" Jul 09 04:41:53.772317 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:53.772277 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a26966701c4f92773d498426164ecdd894d4ab761dc17e8eda98cfee2a49200"} err="failed to get container status \"1a26966701c4f92773d498426164ecdd894d4ab761dc17e8eda98cfee2a49200\": rpc error: code = NotFound desc = could not find container \"1a26966701c4f92773d498426164ecdd894d4ab761dc17e8eda98cfee2a49200\": container with ID starting with 1a26966701c4f92773d498426164ecdd894d4ab761dc17e8eda98cfee2a49200 not found: ID does not exist" Jul 09 04:41:53.778788 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:53.778766 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-5kk9h"] Jul 09 04:41:53.782202 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:53.782184 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-5kk9h"] Jul 09 04:41:53.791513 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:53.791487 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-72kww"] Jul 09 04:41:53.794997 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:53.794977 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-72kww"] Jul 09 04:41:54.195171 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:54.195141 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="215eb8cd-69b0-4731-ad12-3d5d87a509e5" path="/var/lib/kubelet/pods/215eb8cd-69b0-4731-ad12-3d5d87a509e5/volumes" Jul 09 04:41:54.195384 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:54.195372 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d49db64-1440-404b-91c9-607b023fb406" path="/var/lib/kubelet/pods/3d49db64-1440-404b-91c9-607b023fb406/volumes" Jul 09 04:41:54.195663 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:54.195653 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66ca5e10-c082-4577-a83a-0618943eeada" path="/var/lib/kubelet/pods/66ca5e10-c082-4577-a83a-0618943eeada/volumes" Jul 09 04:41:57.065501 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:57.065465 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:41:57.072699 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:57.072672 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6fb67974f9-cldrs"] Jul 09 04:41:57.073009 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:57.072996 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d49db64-1440-404b-91c9-607b023fb406" containerName="authorino" Jul 09 04:41:57.073053 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:57.073012 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d49db64-1440-404b-91c9-607b023fb406" containerName="authorino" Jul 09 04:41:57.073053 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:57.073021 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66ca5e10-c082-4577-a83a-0618943eeada" containerName="authorino" Jul 09 04:41:57.073053 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:57.073027 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="66ca5e10-c082-4577-a83a-0618943eeada" containerName="authorino" Jul 09 04:41:57.073144 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:57.073094 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="66ca5e10-c082-4577-a83a-0618943eeada" containerName="authorino" Jul 09 04:41:57.073144 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:57.073102 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d49db64-1440-404b-91c9-607b023fb406" containerName="authorino" Jul 09 04:41:57.077393 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:57.077374 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fb67974f9-cldrs" Jul 09 04:41:57.080047 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:57.080027 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 09 04:41:57.080149 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:57.080049 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 09 04:41:57.080262 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:57.080250 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-jxlrk\"" Jul 09 04:41:57.085352 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:57.085318 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6fb67974f9-cldrs"] Jul 09 04:41:57.132677 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:57.132644 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mn7mj\" (UniqueName: \"kubernetes.io/projected/0d3e721d-fa01-4c89-b8f9-f7090ed80f81-kube-api-access-mn7mj\") pod \"maas-controller-6fb67974f9-cldrs\" (UID: \"0d3e721d-fa01-4c89-b8f9-f7090ed80f81\") " pod="opendatahub/maas-controller-6fb67974f9-cldrs" Jul 09 04:41:57.132850 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:57.132701 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0d3e721d-fa01-4c89-b8f9-f7090ed80f81-webhook-cert\") pod \"maas-controller-6fb67974f9-cldrs\" (UID: \"0d3e721d-fa01-4c89-b8f9-f7090ed80f81\") " pod="opendatahub/maas-controller-6fb67974f9-cldrs" Jul 09 04:41:57.166192 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:57.166161 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:41:57.233528 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:57.233486 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0d3e721d-fa01-4c89-b8f9-f7090ed80f81-webhook-cert\") pod \"maas-controller-6fb67974f9-cldrs\" (UID: \"0d3e721d-fa01-4c89-b8f9-f7090ed80f81\") " pod="opendatahub/maas-controller-6fb67974f9-cldrs" Jul 09 04:41:57.233731 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:57.233600 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mn7mj\" (UniqueName: \"kubernetes.io/projected/0d3e721d-fa01-4c89-b8f9-f7090ed80f81-kube-api-access-mn7mj\") pod \"maas-controller-6fb67974f9-cldrs\" (UID: \"0d3e721d-fa01-4c89-b8f9-f7090ed80f81\") " pod="opendatahub/maas-controller-6fb67974f9-cldrs" Jul 09 04:41:57.233731 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:41:57.233637 2565 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 09 04:41:57.233731 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:41:57.233710 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d3e721d-fa01-4c89-b8f9-f7090ed80f81-webhook-cert podName:0d3e721d-fa01-4c89-b8f9-f7090ed80f81 nodeName:}" failed. No retries permitted until 2026-07-09 04:41:57.733690839 +0000 UTC m=+589.985214397 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/0d3e721d-fa01-4c89-b8f9-f7090ed80f81-webhook-cert") pod "maas-controller-6fb67974f9-cldrs" (UID: "0d3e721d-fa01-4c89-b8f9-f7090ed80f81") : secret "maas-controller-webhook-cert" not found Jul 09 04:41:57.242828 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:57.242791 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mn7mj\" (UniqueName: \"kubernetes.io/projected/0d3e721d-fa01-4c89-b8f9-f7090ed80f81-kube-api-access-mn7mj\") pod \"maas-controller-6fb67974f9-cldrs\" (UID: \"0d3e721d-fa01-4c89-b8f9-f7090ed80f81\") " pod="opendatahub/maas-controller-6fb67974f9-cldrs" Jul 09 04:41:57.261135 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:57.261105 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:41:57.264991 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:57.264969 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-d78b58cc-bxrcx"] Jul 09 04:41:57.268404 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:57.268388 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d78b58cc-bxrcx" Jul 09 04:41:57.276592 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:57.276572 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d78b58cc-bxrcx"] Jul 09 04:41:57.314182 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:57.314147 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:41:57.334486 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:57.334406 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cae0ebb4-337e-4a10-a754-f567691e1c6e-webhook-cert\") pod \"maas-controller-d78b58cc-bxrcx\" (UID: \"cae0ebb4-337e-4a10-a754-f567691e1c6e\") " pod="opendatahub/maas-controller-d78b58cc-bxrcx" Jul 09 04:41:57.334621 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:57.334558 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nk76f\" (UniqueName: \"kubernetes.io/projected/cae0ebb4-337e-4a10-a754-f567691e1c6e-kube-api-access-nk76f\") pod \"maas-controller-d78b58cc-bxrcx\" (UID: \"cae0ebb4-337e-4a10-a754-f567691e1c6e\") " pod="opendatahub/maas-controller-d78b58cc-bxrcx" Jul 09 04:41:57.435585 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:57.435553 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nk76f\" (UniqueName: \"kubernetes.io/projected/cae0ebb4-337e-4a10-a754-f567691e1c6e-kube-api-access-nk76f\") pod \"maas-controller-d78b58cc-bxrcx\" (UID: \"cae0ebb4-337e-4a10-a754-f567691e1c6e\") " pod="opendatahub/maas-controller-d78b58cc-bxrcx" Jul 09 04:41:57.435730 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:57.435632 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cae0ebb4-337e-4a10-a754-f567691e1c6e-webhook-cert\") pod \"maas-controller-d78b58cc-bxrcx\" (UID: \"cae0ebb4-337e-4a10-a754-f567691e1c6e\") " pod="opendatahub/maas-controller-d78b58cc-bxrcx" Jul 09 04:41:57.437982 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:57.437961 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cae0ebb4-337e-4a10-a754-f567691e1c6e-webhook-cert\") pod \"maas-controller-d78b58cc-bxrcx\" (UID: \"cae0ebb4-337e-4a10-a754-f567691e1c6e\") " pod="opendatahub/maas-controller-d78b58cc-bxrcx" Jul 09 04:41:57.446565 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:57.446545 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nk76f\" (UniqueName: \"kubernetes.io/projected/cae0ebb4-337e-4a10-a754-f567691e1c6e-kube-api-access-nk76f\") pod \"maas-controller-d78b58cc-bxrcx\" (UID: \"cae0ebb4-337e-4a10-a754-f567691e1c6e\") " pod="opendatahub/maas-controller-d78b58cc-bxrcx" Jul 09 04:41:57.579990 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:57.579955 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d78b58cc-bxrcx" Jul 09 04:41:57.716005 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:57.715981 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d78b58cc-bxrcx"] Jul 09 04:41:57.717875 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:41:57.717850 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcae0ebb4_337e_4a10_a754_f567691e1c6e.slice/crio-7856ccbfd706f77bdb2ddb3925d7e2b07d8f69e6dfc318fe52975dd0a63d9810 WatchSource:0}: Error finding container 7856ccbfd706f77bdb2ddb3925d7e2b07d8f69e6dfc318fe52975dd0a63d9810: Status 404 returned error can't find the container with id 7856ccbfd706f77bdb2ddb3925d7e2b07d8f69e6dfc318fe52975dd0a63d9810 Jul 09 04:41:57.738107 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:57.738081 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0d3e721d-fa01-4c89-b8f9-f7090ed80f81-webhook-cert\") pod \"maas-controller-6fb67974f9-cldrs\" (UID: \"0d3e721d-fa01-4c89-b8f9-f7090ed80f81\") " pod="opendatahub/maas-controller-6fb67974f9-cldrs" Jul 09 04:41:57.740323 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:57.740301 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0d3e721d-fa01-4c89-b8f9-f7090ed80f81-webhook-cert\") pod \"maas-controller-6fb67974f9-cldrs\" (UID: \"0d3e721d-fa01-4c89-b8f9-f7090ed80f81\") " pod="opendatahub/maas-controller-6fb67974f9-cldrs" Jul 09 04:41:57.772725 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:57.772696 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d78b58cc-bxrcx" event={"ID":"cae0ebb4-337e-4a10-a754-f567691e1c6e","Type":"ContainerStarted","Data":"7856ccbfd706f77bdb2ddb3925d7e2b07d8f69e6dfc318fe52975dd0a63d9810"} Jul 09 04:41:57.988885 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:57.988847 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fb67974f9-cldrs" Jul 09 04:41:58.118839 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:58.118803 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6fb67974f9-cldrs"] Jul 09 04:41:58.121637 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:41:58.121616 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d3e721d_fa01_4c89_b8f9_f7090ed80f81.slice/crio-2d801a348704294846e8f44f4102d501d43f260160234f1f43547b8290f582b9 WatchSource:0}: Error finding container 2d801a348704294846e8f44f4102d501d43f260160234f1f43547b8290f582b9: Status 404 returned error can't find the container with id 2d801a348704294846e8f44f4102d501d43f260160234f1f43547b8290f582b9 Jul 09 04:41:58.785598 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:41:58.785499 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fb67974f9-cldrs" event={"ID":"0d3e721d-fa01-4c89-b8f9-f7090ed80f81","Type":"ContainerStarted","Data":"2d801a348704294846e8f44f4102d501d43f260160234f1f43547b8290f582b9"} Jul 09 04:42:00.787878 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:00.787753 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:42:00.797258 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:00.797226 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d78b58cc-bxrcx" event={"ID":"cae0ebb4-337e-4a10-a754-f567691e1c6e","Type":"ContainerStarted","Data":"b5b7bfeb045ebc39c1558e15572369bf11387627fd93bfe77266ddad353d9d9f"} Jul 09 04:42:00.797442 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:00.797299 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-d78b58cc-bxrcx" Jul 09 04:42:00.798757 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:00.798731 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fb67974f9-cldrs" event={"ID":"0d3e721d-fa01-4c89-b8f9-f7090ed80f81","Type":"ContainerStarted","Data":"8381144dce3a49750923e83809ae04d0d62990f1c088c12bc44d6497c5e76b52"} Jul 09 04:42:00.798860 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:00.798784 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6fb67974f9-cldrs" Jul 09 04:42:00.816304 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:00.816252 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-d78b58cc-bxrcx" podStartSLOduration=1.003771355 podStartE2EDuration="3.816236428s" totalImagesPullingTime="2.812465073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:41:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:42:00.815254704 +0000 UTC m=+593.066778286" watchObservedRunningTime="2026-07-09 04:42:00.816236428 +0000 UTC m=+593.067760007" Jul 09 04:42:00.833864 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:00.833810 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6fb67974f9-cldrs" podStartSLOduration=1.4179809620000001 podStartE2EDuration="3.833793587s" totalImagesPullingTime="2.415812625s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:41:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:42:00.832734179 +0000 UTC m=+593.084257756" watchObservedRunningTime="2026-07-09 04:42:00.833793587 +0000 UTC m=+593.085317165" Jul 09 04:42:00.873014 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:00.872987 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:42:11.810872 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:11.810840 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6fb67974f9-cldrs" Jul 09 04:42:11.811406 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:11.810910 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-d78b58cc-bxrcx" Jul 09 04:42:11.868115 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:11.868080 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6fb67974f9-cldrs"] Jul 09 04:42:11.868285 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:11.868262 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-6fb67974f9-cldrs" podUID="0d3e721d-fa01-4c89-b8f9-f7090ed80f81" containerName="manager" containerID="cri-o://8381144dce3a49750923e83809ae04d0d62990f1c088c12bc44d6497c5e76b52" gracePeriod=30 Jul 09 04:42:12.113168 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:12.113144 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fb67974f9-cldrs" Jul 09 04:42:12.274201 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:12.274169 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0d3e721d-fa01-4c89-b8f9-f7090ed80f81-webhook-cert\") pod \"0d3e721d-fa01-4c89-b8f9-f7090ed80f81\" (UID: \"0d3e721d-fa01-4c89-b8f9-f7090ed80f81\") " Jul 09 04:42:12.274404 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:12.274255 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mn7mj\" (UniqueName: \"kubernetes.io/projected/0d3e721d-fa01-4c89-b8f9-f7090ed80f81-kube-api-access-mn7mj\") pod \"0d3e721d-fa01-4c89-b8f9-f7090ed80f81\" (UID: \"0d3e721d-fa01-4c89-b8f9-f7090ed80f81\") " Jul 09 04:42:12.276516 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:12.276489 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d3e721d-fa01-4c89-b8f9-f7090ed80f81-kube-api-access-mn7mj" (OuterVolumeSpecName: "kube-api-access-mn7mj") pod "0d3e721d-fa01-4c89-b8f9-f7090ed80f81" (UID: "0d3e721d-fa01-4c89-b8f9-f7090ed80f81"). InnerVolumeSpecName "kube-api-access-mn7mj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:42:12.276628 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:12.276517 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d3e721d-fa01-4c89-b8f9-f7090ed80f81-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "0d3e721d-fa01-4c89-b8f9-f7090ed80f81" (UID: "0d3e721d-fa01-4c89-b8f9-f7090ed80f81"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:42:12.375945 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:12.375855 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mn7mj\" (UniqueName: \"kubernetes.io/projected/0d3e721d-fa01-4c89-b8f9-f7090ed80f81-kube-api-access-mn7mj\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:42:12.375945 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:12.375893 2565 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0d3e721d-fa01-4c89-b8f9-f7090ed80f81-webhook-cert\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:42:12.857156 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:12.856794 2565 generic.go:356] "Generic (PLEG): container finished" podID="0d3e721d-fa01-4c89-b8f9-f7090ed80f81" containerID="8381144dce3a49750923e83809ae04d0d62990f1c088c12bc44d6497c5e76b52" exitCode=0 Jul 09 04:42:12.857156 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:12.856860 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fb67974f9-cldrs" Jul 09 04:42:12.857156 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:12.856920 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fb67974f9-cldrs" event={"ID":"0d3e721d-fa01-4c89-b8f9-f7090ed80f81","Type":"ContainerDied","Data":"8381144dce3a49750923e83809ae04d0d62990f1c088c12bc44d6497c5e76b52"} Jul 09 04:42:12.857156 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:12.856966 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fb67974f9-cldrs" event={"ID":"0d3e721d-fa01-4c89-b8f9-f7090ed80f81","Type":"ContainerDied","Data":"2d801a348704294846e8f44f4102d501d43f260160234f1f43547b8290f582b9"} Jul 09 04:42:12.857156 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:12.856995 2565 scope.go:117] "RemoveContainer" containerID="8381144dce3a49750923e83809ae04d0d62990f1c088c12bc44d6497c5e76b52" Jul 09 04:42:12.867298 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:12.867275 2565 scope.go:117] "RemoveContainer" containerID="8381144dce3a49750923e83809ae04d0d62990f1c088c12bc44d6497c5e76b52" Jul 09 04:42:12.867592 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:42:12.867571 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8381144dce3a49750923e83809ae04d0d62990f1c088c12bc44d6497c5e76b52\": container with ID starting with 8381144dce3a49750923e83809ae04d0d62990f1c088c12bc44d6497c5e76b52 not found: ID does not exist" containerID="8381144dce3a49750923e83809ae04d0d62990f1c088c12bc44d6497c5e76b52" Jul 09 04:42:12.867671 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:12.867606 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8381144dce3a49750923e83809ae04d0d62990f1c088c12bc44d6497c5e76b52"} err="failed to get container status \"8381144dce3a49750923e83809ae04d0d62990f1c088c12bc44d6497c5e76b52\": rpc error: code = NotFound desc = could not find container \"8381144dce3a49750923e83809ae04d0d62990f1c088c12bc44d6497c5e76b52\": container with ID starting with 8381144dce3a49750923e83809ae04d0d62990f1c088c12bc44d6497c5e76b52 not found: ID does not exist" Jul 09 04:42:12.877558 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:12.877529 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6fb67974f9-cldrs"] Jul 09 04:42:12.883233 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:12.883210 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6fb67974f9-cldrs"] Jul 09 04:42:14.195395 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:14.195353 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d3e721d-fa01-4c89-b8f9-f7090ed80f81" path="/var/lib/kubelet/pods/0d3e721d-fa01-4c89-b8f9-f7090ed80f81/volumes" Jul 09 04:42:18.259487 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:18.259452 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:42:18.307448 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:18.307414 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:42:26.699685 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:26.699645 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:42:33.147371 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:33.147267 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:42:33.698891 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:33.698865 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:42:33.894891 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:33.894859 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:42:34.124542 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:34.124458 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:42:34.223986 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:34.223943 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-w9nq6"] Jul 09 04:42:34.224530 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:34.224511 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d3e721d-fa01-4c89-b8f9-f7090ed80f81" containerName="manager" Jul 09 04:42:34.224605 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:34.224534 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d3e721d-fa01-4c89-b8f9-f7090ed80f81" containerName="manager" Jul 09 04:42:34.224664 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:34.224642 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d3e721d-fa01-4c89-b8f9-f7090ed80f81" containerName="manager" Jul 09 04:42:34.225998 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:34.225966 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:42:34.227577 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:34.227559 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-w9nq6" Jul 09 04:42:34.229970 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:34.229952 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-dw62j\"" Jul 09 04:42:34.230289 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:34.230273 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 09 04:42:34.235421 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:34.235396 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-w9nq6"] Jul 09 04:42:34.255991 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:34.255963 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/762233b1-a5ab-46d1-ad87-e1a2c2c4bd05-plugins-config-volume\") pod \"payload-pre-processing-7964457696-w9nq6\" (UID: \"762233b1-a5ab-46d1-ad87-e1a2c2c4bd05\") " pod="openshift-ingress/payload-pre-processing-7964457696-w9nq6" Jul 09 04:42:34.256177 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:34.256041 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56ddr\" (UniqueName: \"kubernetes.io/projected/762233b1-a5ab-46d1-ad87-e1a2c2c4bd05-kube-api-access-56ddr\") pod \"payload-pre-processing-7964457696-w9nq6\" (UID: \"762233b1-a5ab-46d1-ad87-e1a2c2c4bd05\") " pod="openshift-ingress/payload-pre-processing-7964457696-w9nq6" Jul 09 04:42:34.257615 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:34.257592 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-wjbsj"] Jul 09 04:42:34.260990 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:34.260970 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-wjbsj" Jul 09 04:42:34.274478 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:34.274450 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-wjbsj"] Jul 09 04:42:34.357485 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:34.357445 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56ddr\" (UniqueName: \"kubernetes.io/projected/762233b1-a5ab-46d1-ad87-e1a2c2c4bd05-kube-api-access-56ddr\") pod \"payload-pre-processing-7964457696-w9nq6\" (UID: \"762233b1-a5ab-46d1-ad87-e1a2c2c4bd05\") " pod="openshift-ingress/payload-pre-processing-7964457696-w9nq6" Jul 09 04:42:34.357703 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:34.357550 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/762233b1-a5ab-46d1-ad87-e1a2c2c4bd05-plugins-config-volume\") pod \"payload-pre-processing-7964457696-w9nq6\" (UID: \"762233b1-a5ab-46d1-ad87-e1a2c2c4bd05\") " pod="openshift-ingress/payload-pre-processing-7964457696-w9nq6" Jul 09 04:42:34.357703 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:34.357575 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8d4c4c2-a870-4168-ac0e-6281adc1115d-plugins-config-volume\") pod \"payload-processing-58f86c7798-wjbsj\" (UID: \"e8d4c4c2-a870-4168-ac0e-6281adc1115d\") " pod="openshift-ingress/payload-processing-58f86c7798-wjbsj" Jul 09 04:42:34.357703 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:34.357590 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvpzj\" (UniqueName: \"kubernetes.io/projected/e8d4c4c2-a870-4168-ac0e-6281adc1115d-kube-api-access-gvpzj\") pod \"payload-processing-58f86c7798-wjbsj\" (UID: \"e8d4c4c2-a870-4168-ac0e-6281adc1115d\") " pod="openshift-ingress/payload-processing-58f86c7798-wjbsj" Jul 09 04:42:34.358204 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:34.358186 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/762233b1-a5ab-46d1-ad87-e1a2c2c4bd05-plugins-config-volume\") pod \"payload-pre-processing-7964457696-w9nq6\" (UID: \"762233b1-a5ab-46d1-ad87-e1a2c2c4bd05\") " pod="openshift-ingress/payload-pre-processing-7964457696-w9nq6" Jul 09 04:42:34.377948 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:34.377875 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56ddr\" (UniqueName: \"kubernetes.io/projected/762233b1-a5ab-46d1-ad87-e1a2c2c4bd05-kube-api-access-56ddr\") pod \"payload-pre-processing-7964457696-w9nq6\" (UID: \"762233b1-a5ab-46d1-ad87-e1a2c2c4bd05\") " pod="openshift-ingress/payload-pre-processing-7964457696-w9nq6" Jul 09 04:42:34.458043 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:34.458006 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8d4c4c2-a870-4168-ac0e-6281adc1115d-plugins-config-volume\") pod \"payload-processing-58f86c7798-wjbsj\" (UID: \"e8d4c4c2-a870-4168-ac0e-6281adc1115d\") " pod="openshift-ingress/payload-processing-58f86c7798-wjbsj" Jul 09 04:42:34.458221 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:34.458055 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvpzj\" (UniqueName: \"kubernetes.io/projected/e8d4c4c2-a870-4168-ac0e-6281adc1115d-kube-api-access-gvpzj\") pod \"payload-processing-58f86c7798-wjbsj\" (UID: \"e8d4c4c2-a870-4168-ac0e-6281adc1115d\") " pod="openshift-ingress/payload-processing-58f86c7798-wjbsj" Jul 09 04:42:34.458756 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:34.458731 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8d4c4c2-a870-4168-ac0e-6281adc1115d-plugins-config-volume\") pod \"payload-processing-58f86c7798-wjbsj\" (UID: \"e8d4c4c2-a870-4168-ac0e-6281adc1115d\") " pod="openshift-ingress/payload-processing-58f86c7798-wjbsj" Jul 09 04:42:34.469207 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:34.469177 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvpzj\" (UniqueName: \"kubernetes.io/projected/e8d4c4c2-a870-4168-ac0e-6281adc1115d-kube-api-access-gvpzj\") pod \"payload-processing-58f86c7798-wjbsj\" (UID: \"e8d4c4c2-a870-4168-ac0e-6281adc1115d\") " pod="openshift-ingress/payload-processing-58f86c7798-wjbsj" Jul 09 04:42:34.538463 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:34.538426 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-w9nq6" Jul 09 04:42:34.570727 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:34.570691 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-wjbsj" Jul 09 04:42:34.682574 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:34.682538 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-w9nq6"] Jul 09 04:42:34.684991 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:42:34.684965 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod762233b1_a5ab_46d1_ad87_e1a2c2c4bd05.slice/crio-f3099105ba9cb9dc8b72a8a93bc9406acc159930fbf7e1bf547ce94f86002cea WatchSource:0}: Error finding container f3099105ba9cb9dc8b72a8a93bc9406acc159930fbf7e1bf547ce94f86002cea: Status 404 returned error can't find the container with id f3099105ba9cb9dc8b72a8a93bc9406acc159930fbf7e1bf547ce94f86002cea Jul 09 04:42:34.687592 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:34.687569 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 04:42:34.715725 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:34.715702 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-wjbsj"] Jul 09 04:42:34.717763 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:42:34.717738 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8d4c4c2_a870_4168_ac0e_6281adc1115d.slice/crio-7edf6b5174a4aef722f632d1e0cc30738a4f2d33492034b0d3797b79dba87ebd WatchSource:0}: Error finding container 7edf6b5174a4aef722f632d1e0cc30738a4f2d33492034b0d3797b79dba87ebd: Status 404 returned error can't find the container with id 7edf6b5174a4aef722f632d1e0cc30738a4f2d33492034b0d3797b79dba87ebd Jul 09 04:42:34.959871 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:34.959831 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-w9nq6" event={"ID":"762233b1-a5ab-46d1-ad87-e1a2c2c4bd05","Type":"ContainerStarted","Data":"f3099105ba9cb9dc8b72a8a93bc9406acc159930fbf7e1bf547ce94f86002cea"} Jul 09 04:42:34.960980 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:34.960958 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-wjbsj" event={"ID":"e8d4c4c2-a870-4168-ac0e-6281adc1115d","Type":"ContainerStarted","Data":"7edf6b5174a4aef722f632d1e0cc30738a4f2d33492034b0d3797b79dba87ebd"} Jul 09 04:42:36.977468 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:36.977432 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-wjbsj" event={"ID":"e8d4c4c2-a870-4168-ac0e-6281adc1115d","Type":"ContainerStarted","Data":"be55aec2bb900f7b6641546475076551275d0660713f81579fe283f1ff0deb58"} Jul 09 04:42:36.977949 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:36.977494 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-wjbsj" Jul 09 04:42:36.978986 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:36.978961 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-w9nq6" event={"ID":"762233b1-a5ab-46d1-ad87-e1a2c2c4bd05","Type":"ContainerStarted","Data":"84fdce27bc6925aeb2937f9e1bad45e491ab4c7defa4fbfbee19b42f65dd27b9"} Jul 09 04:42:36.979105 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:36.979006 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-w9nq6" Jul 09 04:42:36.999086 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:36.999025 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-wjbsj" podStartSLOduration=1.200329848 podStartE2EDuration="2.999003476s" totalImagesPullingTime="1.798673628s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:42:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:42:36.995181415 +0000 UTC m=+629.246704993" watchObservedRunningTime="2026-07-09 04:42:36.999003476 +0000 UTC m=+629.250527056" Jul 09 04:42:37.011073 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:37.011012 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-w9nq6" podStartSLOduration=1.183079717 podStartE2EDuration="3.010990711s" totalImagesPullingTime="1.827910994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:42:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:42:37.0105926 +0000 UTC m=+629.262116184" watchObservedRunningTime="2026-07-09 04:42:37.010990711 +0000 UTC m=+629.262514292" Jul 09 04:42:42.006945 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:42.006913 2565 generic.go:356] "Generic (PLEG): container finished" podID="ae4bac90-3e35-4e14-ac3d-126c2f7d274b" containerID="ef37413ca06efcb6d3b4cec072e0913ac285d281af18b343bc505835829c90dc" exitCode=0 Jul 09 04:42:42.007466 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:42.006983 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" event={"ID":"ae4bac90-3e35-4e14-ac3d-126c2f7d274b","Type":"ContainerDied","Data":"ef37413ca06efcb6d3b4cec072e0913ac285d281af18b343bc505835829c90dc"} Jul 09 04:42:42.007466 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:42.007385 2565 scope.go:117] "RemoveContainer" containerID="ef37413ca06efcb6d3b4cec072e0913ac285d281af18b343bc505835829c90dc" Jul 09 04:42:42.008333 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:42.008304 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:42:42.008482 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:42.008389 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:42:42.008482 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:42.008421 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 04:42:43.017370 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:43.017317 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" event={"ID":"ae4bac90-3e35-4e14-ac3d-126c2f7d274b","Type":"ContainerStarted","Data":"12c930565a56994b08dc2da99b56b8f8e3779a60a0368c5b03e5c117d2739053"} Jul 09 04:42:43.517822 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:43.517781 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" Jul 09 04:42:43.593006 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:43.592970 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:42:43.696604 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:43.696560 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:42:43.757418 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:43.757385 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" Jul 09 04:42:43.758481 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:43.758456 2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-kwvjl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body= Jul 09 04:42:43.758597 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:43.758519 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" podUID="ae4bac90-3e35-4e14-ac3d-126c2f7d274b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" Jul 09 04:42:44.757334 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:44.757297 2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-kwvjl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body= Jul 09 04:42:44.757712 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:44.757384 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" podUID="ae4bac90-3e35-4e14-ac3d-126c2f7d274b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" Jul 09 04:42:45.758105 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:45.758075 2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-kwvjl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body= Jul 09 04:42:45.758500 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:45.758132 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" podUID="ae4bac90-3e35-4e14-ac3d-126c2f7d274b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" Jul 09 04:42:46.757400 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:46.757363 2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-kwvjl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body= Jul 09 04:42:46.757580 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:46.757430 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" podUID="ae4bac90-3e35-4e14-ac3d-126c2f7d274b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" Jul 09 04:42:47.614648 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:47.614599 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:42:47.761159 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:47.761121 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" Jul 09 04:42:47.762182 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:47.762160 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-kwvjl" Jul 09 04:42:47.985844 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:47.985818 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-w9nq6" Jul 09 04:42:47.986028 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:47.985974 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-wjbsj" Jul 09 04:42:51.350086 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:51.350050 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t"] Jul 09 04:42:51.353111 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:51.353088 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t" Jul 09 04:42:51.356959 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:51.356931 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 09 04:42:51.357151 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:51.357066 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-9g9xn\"" Jul 09 04:42:51.357151 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:51.357089 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 09 04:42:51.357315 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:51.357184 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 09 04:42:51.359557 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:51.359532 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t"] Jul 09 04:42:51.369729 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:51.369703 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:42:51.409904 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:51.409866 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46e18624-cb9b-436b-89f0-2dac0768866e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t\" (UID: \"46e18624-cb9b-436b-89f0-2dac0768866e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t" Jul 09 04:42:51.409904 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:51.409911 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46e18624-cb9b-436b-89f0-2dac0768866e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t\" (UID: \"46e18624-cb9b-436b-89f0-2dac0768866e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t" Jul 09 04:42:51.410099 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:51.409942 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46e18624-cb9b-436b-89f0-2dac0768866e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t\" (UID: \"46e18624-cb9b-436b-89f0-2dac0768866e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t" Jul 09 04:42:51.410099 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:51.410011 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46e18624-cb9b-436b-89f0-2dac0768866e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t\" (UID: \"46e18624-cb9b-436b-89f0-2dac0768866e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t" Jul 09 04:42:51.410099 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:51.410053 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46e18624-cb9b-436b-89f0-2dac0768866e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t\" (UID: \"46e18624-cb9b-436b-89f0-2dac0768866e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t" Jul 09 04:42:51.410099 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:51.410071 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvcpj\" (UniqueName: \"kubernetes.io/projected/46e18624-cb9b-436b-89f0-2dac0768866e-kube-api-access-pvcpj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t\" (UID: \"46e18624-cb9b-436b-89f0-2dac0768866e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t" Jul 09 04:42:51.511048 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:51.511010 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46e18624-cb9b-436b-89f0-2dac0768866e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t\" (UID: \"46e18624-cb9b-436b-89f0-2dac0768866e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t" Jul 09 04:42:51.511048 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:51.511050 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvcpj\" (UniqueName: \"kubernetes.io/projected/46e18624-cb9b-436b-89f0-2dac0768866e-kube-api-access-pvcpj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t\" (UID: \"46e18624-cb9b-436b-89f0-2dac0768866e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t" Jul 09 04:42:51.511269 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:51.511098 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46e18624-cb9b-436b-89f0-2dac0768866e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t\" (UID: \"46e18624-cb9b-436b-89f0-2dac0768866e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t" Jul 09 04:42:51.511269 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:51.511122 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46e18624-cb9b-436b-89f0-2dac0768866e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t\" (UID: \"46e18624-cb9b-436b-89f0-2dac0768866e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t" Jul 09 04:42:51.511269 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:51.511155 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46e18624-cb9b-436b-89f0-2dac0768866e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t\" (UID: \"46e18624-cb9b-436b-89f0-2dac0768866e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t" Jul 09 04:42:51.511269 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:51.511199 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46e18624-cb9b-436b-89f0-2dac0768866e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t\" (UID: \"46e18624-cb9b-436b-89f0-2dac0768866e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t" Jul 09 04:42:51.511497 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:51.511475 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46e18624-cb9b-436b-89f0-2dac0768866e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t\" (UID: \"46e18624-cb9b-436b-89f0-2dac0768866e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t" Jul 09 04:42:51.511586 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:51.511565 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46e18624-cb9b-436b-89f0-2dac0768866e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t\" (UID: \"46e18624-cb9b-436b-89f0-2dac0768866e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t" Jul 09 04:42:51.511636 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:51.511611 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46e18624-cb9b-436b-89f0-2dac0768866e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t\" (UID: \"46e18624-cb9b-436b-89f0-2dac0768866e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t" Jul 09 04:42:51.513350 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:51.513317 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46e18624-cb9b-436b-89f0-2dac0768866e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t\" (UID: \"46e18624-cb9b-436b-89f0-2dac0768866e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t" Jul 09 04:42:51.513618 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:51.513601 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46e18624-cb9b-436b-89f0-2dac0768866e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t\" (UID: \"46e18624-cb9b-436b-89f0-2dac0768866e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t" Jul 09 04:42:51.519803 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:51.519772 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvcpj\" (UniqueName: \"kubernetes.io/projected/46e18624-cb9b-436b-89f0-2dac0768866e-kube-api-access-pvcpj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t\" (UID: \"46e18624-cb9b-436b-89f0-2dac0768866e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t" Jul 09 04:42:51.665737 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:51.665702 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t" Jul 09 04:42:51.798957 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:51.798921 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t"] Jul 09 04:42:51.802678 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:42:51.802639 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46e18624_cb9b_436b_89f0_2dac0768866e.slice/crio-e4e2a8713905643cbd522e2a9f1df4b8cb247529fa6d90661ce02e4bad85e491 WatchSource:0}: Error finding container e4e2a8713905643cbd522e2a9f1df4b8cb247529fa6d90661ce02e4bad85e491: Status 404 returned error can't find the container with id e4e2a8713905643cbd522e2a9f1df4b8cb247529fa6d90661ce02e4bad85e491 Jul 09 04:42:52.067043 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:52.066956 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t" event={"ID":"46e18624-cb9b-436b-89f0-2dac0768866e","Type":"ContainerStarted","Data":"e4e2a8713905643cbd522e2a9f1df4b8cb247529fa6d90661ce02e4bad85e491"} Jul 09 04:42:53.078896 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:53.078855 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t" event={"ID":"46e18624-cb9b-436b-89f0-2dac0768866e","Type":"ContainerStarted","Data":"86ea0db6c1b9d410fe5214fd269be47d2386f45b1b78daf30730592f8fa84119"} Jul 09 04:42:53.095760 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:42:53.095704 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t" podStartSLOduration=0.983057919 podStartE2EDuration="2.095688658s" totalImagesPullingTime="1.112630739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:42:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:42:53.094409513 +0000 UTC m=+645.345933091" watchObservedRunningTime="2026-07-09 04:42:53.095688658 +0000 UTC m=+645.347212237" Jul 09 04:43:00.758655 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:00.758612 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-nc8tp"] Jul 09 04:43:00.761544 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:00.761522 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc8tp" Jul 09 04:43:00.764168 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:00.764144 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 09 04:43:00.772627 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:00.772602 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-nc8tp"] Jul 09 04:43:00.790213 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:00.790185 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:43:00.894934 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:00.894890 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bc566c7-fdb5-427c-8718-2b5cd9a58998-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nc8tp\" (UID: \"1bc566c7-fdb5-427c-8718-2b5cd9a58998\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc8tp" Jul 09 04:43:00.894934 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:00.894939 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1bc566c7-fdb5-427c-8718-2b5cd9a58998-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nc8tp\" (UID: \"1bc566c7-fdb5-427c-8718-2b5cd9a58998\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc8tp" Jul 09 04:43:00.895211 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:00.895025 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1bc566c7-fdb5-427c-8718-2b5cd9a58998-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nc8tp\" (UID: \"1bc566c7-fdb5-427c-8718-2b5cd9a58998\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc8tp" Jul 09 04:43:00.895211 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:00.895053 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1bc566c7-fdb5-427c-8718-2b5cd9a58998-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nc8tp\" (UID: \"1bc566c7-fdb5-427c-8718-2b5cd9a58998\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc8tp" Jul 09 04:43:00.895211 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:00.895077 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1bc566c7-fdb5-427c-8718-2b5cd9a58998-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nc8tp\" (UID: \"1bc566c7-fdb5-427c-8718-2b5cd9a58998\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc8tp" Jul 09 04:43:00.895211 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:00.895107 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhfhz\" (UniqueName: \"kubernetes.io/projected/1bc566c7-fdb5-427c-8718-2b5cd9a58998-kube-api-access-lhfhz\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nc8tp\" (UID: \"1bc566c7-fdb5-427c-8718-2b5cd9a58998\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc8tp" Jul 09 04:43:00.995901 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:00.995863 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhfhz\" (UniqueName: \"kubernetes.io/projected/1bc566c7-fdb5-427c-8718-2b5cd9a58998-kube-api-access-lhfhz\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nc8tp\" (UID: \"1bc566c7-fdb5-427c-8718-2b5cd9a58998\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc8tp" Jul 09 04:43:00.996079 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:00.995921 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bc566c7-fdb5-427c-8718-2b5cd9a58998-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nc8tp\" (UID: \"1bc566c7-fdb5-427c-8718-2b5cd9a58998\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc8tp" Jul 09 04:43:00.996079 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:00.995965 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1bc566c7-fdb5-427c-8718-2b5cd9a58998-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nc8tp\" (UID: \"1bc566c7-fdb5-427c-8718-2b5cd9a58998\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc8tp" Jul 09 04:43:00.996155 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:00.996112 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1bc566c7-fdb5-427c-8718-2b5cd9a58998-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nc8tp\" (UID: \"1bc566c7-fdb5-427c-8718-2b5cd9a58998\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc8tp" Jul 09 04:43:00.996209 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:00.996169 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1bc566c7-fdb5-427c-8718-2b5cd9a58998-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nc8tp\" (UID: \"1bc566c7-fdb5-427c-8718-2b5cd9a58998\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc8tp" Jul 09 04:43:00.996209 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:00.996193 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1bc566c7-fdb5-427c-8718-2b5cd9a58998-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nc8tp\" (UID: \"1bc566c7-fdb5-427c-8718-2b5cd9a58998\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc8tp" Jul 09 04:43:00.996464 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:00.996444 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bc566c7-fdb5-427c-8718-2b5cd9a58998-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nc8tp\" (UID: \"1bc566c7-fdb5-427c-8718-2b5cd9a58998\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc8tp" Jul 09 04:43:00.996464 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:00.996457 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1bc566c7-fdb5-427c-8718-2b5cd9a58998-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nc8tp\" (UID: \"1bc566c7-fdb5-427c-8718-2b5cd9a58998\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc8tp" Jul 09 04:43:00.996576 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:00.996545 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1bc566c7-fdb5-427c-8718-2b5cd9a58998-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nc8tp\" (UID: \"1bc566c7-fdb5-427c-8718-2b5cd9a58998\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc8tp" Jul 09 04:43:00.998374 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:00.998351 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1bc566c7-fdb5-427c-8718-2b5cd9a58998-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nc8tp\" (UID: \"1bc566c7-fdb5-427c-8718-2b5cd9a58998\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc8tp" Jul 09 04:43:00.998550 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:00.998532 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1bc566c7-fdb5-427c-8718-2b5cd9a58998-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nc8tp\" (UID: \"1bc566c7-fdb5-427c-8718-2b5cd9a58998\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc8tp" Jul 09 04:43:01.003295 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:01.003278 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhfhz\" (UniqueName: \"kubernetes.io/projected/1bc566c7-fdb5-427c-8718-2b5cd9a58998-kube-api-access-lhfhz\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nc8tp\" (UID: \"1bc566c7-fdb5-427c-8718-2b5cd9a58998\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc8tp" Jul 09 04:43:01.074694 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:01.074583 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc8tp" Jul 09 04:43:01.237284 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:01.237260 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-nc8tp"] Jul 09 04:43:01.239187 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:43:01.239159 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bc566c7_fdb5_427c_8718_2b5cd9a58998.slice/crio-acd1304b5362f1883d368f041f6860711d02382de620bbd191f4c84db8b05611 WatchSource:0}: Error finding container acd1304b5362f1883d368f041f6860711d02382de620bbd191f4c84db8b05611: Status 404 returned error can't find the container with id acd1304b5362f1883d368f041f6860711d02382de620bbd191f4c84db8b05611 Jul 09 04:43:01.666487 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:01.666397 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t" Jul 09 04:43:01.666487 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:01.666468 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t" Jul 09 04:43:01.679060 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:01.679031 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t" Jul 09 04:43:02.124483 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:02.124379 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc8tp" event={"ID":"1bc566c7-fdb5-427c-8718-2b5cd9a58998","Type":"ContainerStarted","Data":"276ca614875429b64b0f1091058fbab486a6fccd8d10bf01758281e10580b582"} Jul 09 04:43:02.124483 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:02.124452 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc8tp" event={"ID":"1bc566c7-fdb5-427c-8718-2b5cd9a58998","Type":"ContainerStarted","Data":"acd1304b5362f1883d368f041f6860711d02382de620bbd191f4c84db8b05611"} Jul 09 04:43:02.141716 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:02.141676 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t" Jul 09 04:43:02.145064 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:02.145004 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc8tp" podStartSLOduration=1.974036546 podStartE2EDuration="2.144982513s" totalImagesPullingTime="170.945967ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:43:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:43:02.144358704 +0000 UTC m=+654.395882282" watchObservedRunningTime="2026-07-09 04:43:02.144982513 +0000 UTC m=+654.396506091" Jul 09 04:43:05.266380 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:05.266325 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:43:11.074725 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:11.074686 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc8tp" Jul 09 04:43:11.075138 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:11.074773 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc8tp" Jul 09 04:43:11.087380 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:11.087329 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc8tp" Jul 09 04:43:11.176829 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:11.176800 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nc8tp" Jul 09 04:43:20.055315 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:20.055280 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:43:33.103943 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:33.103897 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:43:33.154135 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:33.154085 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:43:37.851907 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:37.851872 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-4hnxm"] Jul 09 04:43:37.856162 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:37.856138 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hnxm" Jul 09 04:43:37.858635 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:37.858605 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 09 04:43:37.865722 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:37.865693 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-4hnxm"] Jul 09 04:43:37.887406 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:37.887375 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:43:37.944605 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:37.944570 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxkjg\" (UniqueName: \"kubernetes.io/projected/db73bba3-8398-4c4f-b2fc-b90d8d28e66a-kube-api-access-pxkjg\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4hnxm\" (UID: \"db73bba3-8398-4c4f-b2fc-b90d8d28e66a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hnxm" Jul 09 04:43:37.944782 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:37.944621 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db73bba3-8398-4c4f-b2fc-b90d8d28e66a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4hnxm\" (UID: \"db73bba3-8398-4c4f-b2fc-b90d8d28e66a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hnxm" Jul 09 04:43:37.944782 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:37.944651 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db73bba3-8398-4c4f-b2fc-b90d8d28e66a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4hnxm\" (UID: \"db73bba3-8398-4c4f-b2fc-b90d8d28e66a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hnxm" Jul 09 04:43:37.944782 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:37.944669 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db73bba3-8398-4c4f-b2fc-b90d8d28e66a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4hnxm\" (UID: \"db73bba3-8398-4c4f-b2fc-b90d8d28e66a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hnxm" Jul 09 04:43:37.944782 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:37.944708 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db73bba3-8398-4c4f-b2fc-b90d8d28e66a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4hnxm\" (UID: \"db73bba3-8398-4c4f-b2fc-b90d8d28e66a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hnxm" Jul 09 04:43:37.944929 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:37.944814 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db73bba3-8398-4c4f-b2fc-b90d8d28e66a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4hnxm\" (UID: \"db73bba3-8398-4c4f-b2fc-b90d8d28e66a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hnxm" Jul 09 04:43:38.045948 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:38.045903 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db73bba3-8398-4c4f-b2fc-b90d8d28e66a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4hnxm\" (UID: \"db73bba3-8398-4c4f-b2fc-b90d8d28e66a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hnxm" Jul 09 04:43:38.046141 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:38.045957 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db73bba3-8398-4c4f-b2fc-b90d8d28e66a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4hnxm\" (UID: \"db73bba3-8398-4c4f-b2fc-b90d8d28e66a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hnxm" Jul 09 04:43:38.046141 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:38.046015 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxkjg\" (UniqueName: \"kubernetes.io/projected/db73bba3-8398-4c4f-b2fc-b90d8d28e66a-kube-api-access-pxkjg\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4hnxm\" (UID: \"db73bba3-8398-4c4f-b2fc-b90d8d28e66a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hnxm" Jul 09 04:43:38.046141 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:38.046066 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db73bba3-8398-4c4f-b2fc-b90d8d28e66a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4hnxm\" (UID: \"db73bba3-8398-4c4f-b2fc-b90d8d28e66a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hnxm" Jul 09 04:43:38.046141 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:38.046095 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db73bba3-8398-4c4f-b2fc-b90d8d28e66a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4hnxm\" (UID: \"db73bba3-8398-4c4f-b2fc-b90d8d28e66a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hnxm" Jul 09 04:43:38.046141 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:38.046121 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db73bba3-8398-4c4f-b2fc-b90d8d28e66a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4hnxm\" (UID: \"db73bba3-8398-4c4f-b2fc-b90d8d28e66a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hnxm" Jul 09 04:43:38.046514 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:38.046483 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db73bba3-8398-4c4f-b2fc-b90d8d28e66a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4hnxm\" (UID: \"db73bba3-8398-4c4f-b2fc-b90d8d28e66a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hnxm" Jul 09 04:43:38.046586 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:38.046540 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db73bba3-8398-4c4f-b2fc-b90d8d28e66a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4hnxm\" (UID: \"db73bba3-8398-4c4f-b2fc-b90d8d28e66a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hnxm" Jul 09 04:43:38.046586 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:38.046558 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db73bba3-8398-4c4f-b2fc-b90d8d28e66a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4hnxm\" (UID: \"db73bba3-8398-4c4f-b2fc-b90d8d28e66a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hnxm" Jul 09 04:43:38.048421 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:38.048396 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db73bba3-8398-4c4f-b2fc-b90d8d28e66a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4hnxm\" (UID: \"db73bba3-8398-4c4f-b2fc-b90d8d28e66a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hnxm" Jul 09 04:43:38.048610 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:38.048591 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db73bba3-8398-4c4f-b2fc-b90d8d28e66a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4hnxm\" (UID: \"db73bba3-8398-4c4f-b2fc-b90d8d28e66a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hnxm" Jul 09 04:43:38.053749 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:38.053725 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxkjg\" (UniqueName: \"kubernetes.io/projected/db73bba3-8398-4c4f-b2fc-b90d8d28e66a-kube-api-access-pxkjg\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4hnxm\" (UID: \"db73bba3-8398-4c4f-b2fc-b90d8d28e66a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hnxm" Jul 09 04:43:38.170308 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:38.170276 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hnxm" Jul 09 04:43:38.305293 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:38.305268 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-4hnxm"] Jul 09 04:43:38.308436 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:43:38.308407 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb73bba3_8398_4c4f_b2fc_b90d8d28e66a.slice/crio-80c715b80e59292d30b9134896792d5b94a5cfea488583c8050afe555c7631aa WatchSource:0}: Error finding container 80c715b80e59292d30b9134896792d5b94a5cfea488583c8050afe555c7631aa: Status 404 returned error can't find the container with id 80c715b80e59292d30b9134896792d5b94a5cfea488583c8050afe555c7631aa Jul 09 04:43:39.303300 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:39.303257 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hnxm" event={"ID":"db73bba3-8398-4c4f-b2fc-b90d8d28e66a","Type":"ContainerStarted","Data":"9b4ec4ecdfc80f2c48539e01ef054b2d7bba3a4f5de648e3a1e617eb63857a41"} Jul 09 04:43:39.303701 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:39.303320 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hnxm" event={"ID":"db73bba3-8398-4c4f-b2fc-b90d8d28e66a","Type":"ContainerStarted","Data":"80c715b80e59292d30b9134896792d5b94a5cfea488583c8050afe555c7631aa"} Jul 09 04:43:39.320745 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:39.320696 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hnxm" podStartSLOduration=2.143804522 podStartE2EDuration="2.320681195s" totalImagesPullingTime="176.876673ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:43:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:43:39.319300605 +0000 UTC m=+691.570824184" watchObservedRunningTime="2026-07-09 04:43:39.320681195 +0000 UTC m=+691.572204787" Jul 09 04:43:44.446015 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:44.445978 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk"] Jul 09 04:43:44.449996 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:44.449974 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk" Jul 09 04:43:44.452283 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:44.452258 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 09 04:43:44.459105 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:44.459077 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk"] Jul 09 04:43:44.468766 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:44.468731 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:43:44.607680 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:44.607647 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69c53279-0331-4154-a29d-68b283c0e741-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk\" (UID: \"69c53279-0331-4154-a29d-68b283c0e741\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk" Jul 09 04:43:44.607680 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:44.607682 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhn2v\" (UniqueName: \"kubernetes.io/projected/69c53279-0331-4154-a29d-68b283c0e741-kube-api-access-dhn2v\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk\" (UID: \"69c53279-0331-4154-a29d-68b283c0e741\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk" Jul 09 04:43:44.607911 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:44.607722 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69c53279-0331-4154-a29d-68b283c0e741-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk\" (UID: \"69c53279-0331-4154-a29d-68b283c0e741\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk" Jul 09 04:43:44.607911 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:44.607765 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69c53279-0331-4154-a29d-68b283c0e741-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk\" (UID: \"69c53279-0331-4154-a29d-68b283c0e741\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk" Jul 09 04:43:44.607911 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:44.607794 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69c53279-0331-4154-a29d-68b283c0e741-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk\" (UID: \"69c53279-0331-4154-a29d-68b283c0e741\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk" Jul 09 04:43:44.607911 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:44.607811 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69c53279-0331-4154-a29d-68b283c0e741-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk\" (UID: \"69c53279-0331-4154-a29d-68b283c0e741\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk" Jul 09 04:43:44.708666 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:44.708576 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69c53279-0331-4154-a29d-68b283c0e741-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk\" (UID: \"69c53279-0331-4154-a29d-68b283c0e741\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk" Jul 09 04:43:44.708666 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:44.708615 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhn2v\" (UniqueName: \"kubernetes.io/projected/69c53279-0331-4154-a29d-68b283c0e741-kube-api-access-dhn2v\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk\" (UID: \"69c53279-0331-4154-a29d-68b283c0e741\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk" Jul 09 04:43:44.708849 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:44.708785 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69c53279-0331-4154-a29d-68b283c0e741-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk\" (UID: \"69c53279-0331-4154-a29d-68b283c0e741\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk" Jul 09 04:43:44.708892 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:44.708867 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69c53279-0331-4154-a29d-68b283c0e741-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk\" (UID: \"69c53279-0331-4154-a29d-68b283c0e741\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk" Jul 09 04:43:44.708936 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:44.708917 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69c53279-0331-4154-a29d-68b283c0e741-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk\" (UID: \"69c53279-0331-4154-a29d-68b283c0e741\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk" Jul 09 04:43:44.709032 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:44.709006 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69c53279-0331-4154-a29d-68b283c0e741-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk\" (UID: \"69c53279-0331-4154-a29d-68b283c0e741\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk" Jul 09 04:43:44.709180 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:44.709036 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69c53279-0331-4154-a29d-68b283c0e741-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk\" (UID: \"69c53279-0331-4154-a29d-68b283c0e741\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk" Jul 09 04:43:44.709261 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:44.709195 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69c53279-0331-4154-a29d-68b283c0e741-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk\" (UID: \"69c53279-0331-4154-a29d-68b283c0e741\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk" Jul 09 04:43:44.709261 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:44.709208 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69c53279-0331-4154-a29d-68b283c0e741-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk\" (UID: \"69c53279-0331-4154-a29d-68b283c0e741\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk" Jul 09 04:43:44.711260 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:44.711237 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69c53279-0331-4154-a29d-68b283c0e741-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk\" (UID: \"69c53279-0331-4154-a29d-68b283c0e741\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk" Jul 09 04:43:44.711356 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:44.711301 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69c53279-0331-4154-a29d-68b283c0e741-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk\" (UID: \"69c53279-0331-4154-a29d-68b283c0e741\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk" Jul 09 04:43:44.715916 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:44.715895 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhn2v\" (UniqueName: \"kubernetes.io/projected/69c53279-0331-4154-a29d-68b283c0e741-kube-api-access-dhn2v\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk\" (UID: \"69c53279-0331-4154-a29d-68b283c0e741\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk" Jul 09 04:43:44.761119 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:44.761080 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk" Jul 09 04:43:44.892312 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:44.892288 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk"] Jul 09 04:43:44.894555 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:43:44.894520 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod69c53279_0331_4154_a29d_68b283c0e741.slice/crio-5ec0ffe562c7a963851918dd184b0a320d4a057a7ea1bdb34758f5ac5639ac69 WatchSource:0}: Error finding container 5ec0ffe562c7a963851918dd184b0a320d4a057a7ea1bdb34758f5ac5639ac69: Status 404 returned error can't find the container with id 5ec0ffe562c7a963851918dd184b0a320d4a057a7ea1bdb34758f5ac5639ac69 Jul 09 04:43:45.333473 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:45.333386 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk" event={"ID":"69c53279-0331-4154-a29d-68b283c0e741","Type":"ContainerStarted","Data":"f8cbb82814cf3822886f11c1221df996a0afa32314f473147ac8c47fa981b573"} Jul 09 04:43:45.333473 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:45.333433 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk" event={"ID":"69c53279-0331-4154-a29d-68b283c0e741","Type":"ContainerStarted","Data":"5ec0ffe562c7a963851918dd184b0a320d4a057a7ea1bdb34758f5ac5639ac69"} Jul 09 04:43:45.351033 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:45.350984 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk" podStartSLOduration=1.192528838 podStartE2EDuration="1.350967945s" totalImagesPullingTime="158.439107ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:43:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:43:45.349062389 +0000 UTC m=+697.600585967" watchObservedRunningTime="2026-07-09 04:43:45.350967945 +0000 UTC m=+697.602491523" Jul 09 04:43:48.170585 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:48.170541 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hnxm" Jul 09 04:43:48.170980 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:48.170613 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hnxm" Jul 09 04:43:48.183322 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:48.183300 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hnxm" Jul 09 04:43:48.357537 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:48.357511 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4hnxm" Jul 09 04:43:54.761255 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:54.761220 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk" Jul 09 04:43:54.762144 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:54.761272 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk" Jul 09 04:43:54.773502 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:54.773474 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk" Jul 09 04:43:55.399952 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:43:55.399928 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk" Jul 09 04:44:18.903408 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:18.903372 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5b5d8996bc-hwqqk"] Jul 09 04:44:18.906910 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:18.906891 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5b5d8996bc-hwqqk" Jul 09 04:44:18.912827 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:18.912798 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5b5d8996bc-hwqqk"] Jul 09 04:44:19.023634 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:19.023597 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnslr\" (UniqueName: \"kubernetes.io/projected/e79be3a5-b853-4c59-95c6-7a914490b429-kube-api-access-pnslr\") pod \"authorino-5b5d8996bc-hwqqk\" (UID: \"e79be3a5-b853-4c59-95c6-7a914490b429\") " pod="kuadrant-system/authorino-5b5d8996bc-hwqqk" Jul 09 04:44:19.023808 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:19.023683 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e79be3a5-b853-4c59-95c6-7a914490b429-tls-cert\") pod \"authorino-5b5d8996bc-hwqqk\" (UID: \"e79be3a5-b853-4c59-95c6-7a914490b429\") " pod="kuadrant-system/authorino-5b5d8996bc-hwqqk" Jul 09 04:44:19.124768 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:19.124729 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnslr\" (UniqueName: \"kubernetes.io/projected/e79be3a5-b853-4c59-95c6-7a914490b429-kube-api-access-pnslr\") pod \"authorino-5b5d8996bc-hwqqk\" (UID: \"e79be3a5-b853-4c59-95c6-7a914490b429\") " pod="kuadrant-system/authorino-5b5d8996bc-hwqqk" Jul 09 04:44:19.124937 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:19.124790 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e79be3a5-b853-4c59-95c6-7a914490b429-tls-cert\") pod \"authorino-5b5d8996bc-hwqqk\" (UID: \"e79be3a5-b853-4c59-95c6-7a914490b429\") " pod="kuadrant-system/authorino-5b5d8996bc-hwqqk" Jul 09 04:44:19.127225 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:19.127204 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e79be3a5-b853-4c59-95c6-7a914490b429-tls-cert\") pod \"authorino-5b5d8996bc-hwqqk\" (UID: \"e79be3a5-b853-4c59-95c6-7a914490b429\") " pod="kuadrant-system/authorino-5b5d8996bc-hwqqk" Jul 09 04:44:19.132629 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:19.132610 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnslr\" (UniqueName: \"kubernetes.io/projected/e79be3a5-b853-4c59-95c6-7a914490b429-kube-api-access-pnslr\") pod \"authorino-5b5d8996bc-hwqqk\" (UID: \"e79be3a5-b853-4c59-95c6-7a914490b429\") " pod="kuadrant-system/authorino-5b5d8996bc-hwqqk" Jul 09 04:44:19.216979 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:19.216943 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5b5d8996bc-hwqqk" Jul 09 04:44:19.281907 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:19.281860 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:44:19.360773 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:19.360746 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5b5d8996bc-hwqqk"] Jul 09 04:44:19.362953 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:44:19.362920 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode79be3a5_b853_4c59_95c6_7a914490b429.slice/crio-96cfdbea5e758d6d4a4c6424c3c943124e624cbff4b12108a1b5b67aece77631 WatchSource:0}: Error finding container 96cfdbea5e758d6d4a4c6424c3c943124e624cbff4b12108a1b5b67aece77631: Status 404 returned error can't find the container with id 96cfdbea5e758d6d4a4c6424c3c943124e624cbff4b12108a1b5b67aece77631 Jul 09 04:44:19.363215 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:19.363193 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:44:19.501902 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:19.501813 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5b5d8996bc-hwqqk" event={"ID":"e79be3a5-b853-4c59-95c6-7a914490b429","Type":"ContainerStarted","Data":"96cfdbea5e758d6d4a4c6424c3c943124e624cbff4b12108a1b5b67aece77631"} Jul 09 04:44:20.361688 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:20.361651 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:44:20.458146 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:20.458111 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:44:20.509968 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:20.509918 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5b5d8996bc-hwqqk" event={"ID":"e79be3a5-b853-4c59-95c6-7a914490b429","Type":"ContainerStarted","Data":"fbf1adb9abef40f69c1a3c398a7ca11376dd5cb59b9a1c1b616b932f6bce3913"} Jul 09 04:44:20.529562 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:20.529511 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5b5d8996bc-hwqqk" podStartSLOduration=2.004393005 podStartE2EDuration="2.529494584s" totalImagesPullingTime="525.101579ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:44:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:44:20.527966446 +0000 UTC m=+732.779490025" watchObservedRunningTime="2026-07-09 04:44:20.529494584 +0000 UTC m=+732.781018192" Jul 09 04:44:20.565195 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:20.565159 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5fbd8d9b6-58hdv"] Jul 09 04:44:20.565433 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:20.565397 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5fbd8d9b6-58hdv" podUID="dd0e0e40-a52a-449d-b358-cabef23533e2" containerName="authorino" containerID="cri-o://ebcc204a17da45eb4db2ee9c23d10988b3463c39d243d87fcd66e62182f8a07b" gracePeriod=30 Jul 09 04:44:20.837559 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:20.837534 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5fbd8d9b6-58hdv" Jul 09 04:44:20.941651 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:20.941566 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dd0e0e40-a52a-449d-b358-cabef23533e2-tls-cert\") pod \"dd0e0e40-a52a-449d-b358-cabef23533e2\" (UID: \"dd0e0e40-a52a-449d-b358-cabef23533e2\") " Jul 09 04:44:20.941815 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:20.941660 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9rz62\" (UniqueName: \"kubernetes.io/projected/dd0e0e40-a52a-449d-b358-cabef23533e2-kube-api-access-9rz62\") pod \"dd0e0e40-a52a-449d-b358-cabef23533e2\" (UID: \"dd0e0e40-a52a-449d-b358-cabef23533e2\") " Jul 09 04:44:20.943738 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:20.943701 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd0e0e40-a52a-449d-b358-cabef23533e2-kube-api-access-9rz62" (OuterVolumeSpecName: "kube-api-access-9rz62") pod "dd0e0e40-a52a-449d-b358-cabef23533e2" (UID: "dd0e0e40-a52a-449d-b358-cabef23533e2"). InnerVolumeSpecName "kube-api-access-9rz62". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:44:20.952906 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:20.952873 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd0e0e40-a52a-449d-b358-cabef23533e2-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "dd0e0e40-a52a-449d-b358-cabef23533e2" (UID: "dd0e0e40-a52a-449d-b358-cabef23533e2"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:44:21.043255 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:21.043214 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rz62\" (UniqueName: \"kubernetes.io/projected/dd0e0e40-a52a-449d-b358-cabef23533e2-kube-api-access-9rz62\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:44:21.043255 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:21.043247 2565 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dd0e0e40-a52a-449d-b358-cabef23533e2-tls-cert\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:44:21.517232 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:21.517194 2565 generic.go:356] "Generic (PLEG): container finished" podID="dd0e0e40-a52a-449d-b358-cabef23533e2" containerID="ebcc204a17da45eb4db2ee9c23d10988b3463c39d243d87fcd66e62182f8a07b" exitCode=0 Jul 09 04:44:21.517663 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:21.517240 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5fbd8d9b6-58hdv" Jul 09 04:44:21.517663 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:21.517280 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5fbd8d9b6-58hdv" event={"ID":"dd0e0e40-a52a-449d-b358-cabef23533e2","Type":"ContainerDied","Data":"ebcc204a17da45eb4db2ee9c23d10988b3463c39d243d87fcd66e62182f8a07b"} Jul 09 04:44:21.517663 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:21.517350 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5fbd8d9b6-58hdv" event={"ID":"dd0e0e40-a52a-449d-b358-cabef23533e2","Type":"ContainerDied","Data":"f7226038ee208057150f5e54161292a2989b6a98a8c1eeebb3024a1d605284c9"} Jul 09 04:44:21.517663 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:21.517381 2565 scope.go:117] "RemoveContainer" containerID="ebcc204a17da45eb4db2ee9c23d10988b3463c39d243d87fcd66e62182f8a07b" Jul 09 04:44:21.526962 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:21.526932 2565 scope.go:117] "RemoveContainer" containerID="ebcc204a17da45eb4db2ee9c23d10988b3463c39d243d87fcd66e62182f8a07b" Jul 09 04:44:21.527234 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:44:21.527214 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebcc204a17da45eb4db2ee9c23d10988b3463c39d243d87fcd66e62182f8a07b\": container with ID starting with ebcc204a17da45eb4db2ee9c23d10988b3463c39d243d87fcd66e62182f8a07b not found: ID does not exist" containerID="ebcc204a17da45eb4db2ee9c23d10988b3463c39d243d87fcd66e62182f8a07b" Jul 09 04:44:21.527285 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:21.527243 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebcc204a17da45eb4db2ee9c23d10988b3463c39d243d87fcd66e62182f8a07b"} err="failed to get container status \"ebcc204a17da45eb4db2ee9c23d10988b3463c39d243d87fcd66e62182f8a07b\": rpc error: code = NotFound desc = could not find container \"ebcc204a17da45eb4db2ee9c23d10988b3463c39d243d87fcd66e62182f8a07b\": container with ID starting with ebcc204a17da45eb4db2ee9c23d10988b3463c39d243d87fcd66e62182f8a07b not found: ID does not exist" Jul 09 04:44:21.539826 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:21.539790 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5fbd8d9b6-58hdv"] Jul 09 04:44:21.541558 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:21.541537 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5fbd8d9b6-58hdv"] Jul 09 04:44:22.203225 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:22.200082 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd0e0e40-a52a-449d-b358-cabef23533e2" path="/var/lib/kubelet/pods/dd0e0e40-a52a-449d-b358-cabef23533e2/volumes" Jul 09 04:44:32.194456 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:32.194424 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:44:39.826841 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:39.826805 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:44:39.879970 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:44:39.879940 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:45:30.048312 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:30.048227 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:45:30.143530 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:30.143492 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d78b58cc-bxrcx"] Jul 09 04:45:30.143771 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:30.143725 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-d78b58cc-bxrcx" podUID="cae0ebb4-337e-4a10-a754-f567691e1c6e" containerName="manager" containerID="cri-o://b5b7bfeb045ebc39c1558e15572369bf11387627fd93bfe77266ddad353d9d9f" gracePeriod=30 Jul 09 04:45:30.395131 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:30.395106 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d78b58cc-bxrcx" Jul 09 04:45:30.505441 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:30.505406 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nk76f\" (UniqueName: \"kubernetes.io/projected/cae0ebb4-337e-4a10-a754-f567691e1c6e-kube-api-access-nk76f\") pod \"cae0ebb4-337e-4a10-a754-f567691e1c6e\" (UID: \"cae0ebb4-337e-4a10-a754-f567691e1c6e\") " Jul 09 04:45:30.505626 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:30.505451 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cae0ebb4-337e-4a10-a754-f567691e1c6e-webhook-cert\") pod \"cae0ebb4-337e-4a10-a754-f567691e1c6e\" (UID: \"cae0ebb4-337e-4a10-a754-f567691e1c6e\") " Jul 09 04:45:30.507476 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:30.507444 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cae0ebb4-337e-4a10-a754-f567691e1c6e-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "cae0ebb4-337e-4a10-a754-f567691e1c6e" (UID: "cae0ebb4-337e-4a10-a754-f567691e1c6e"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:45:30.507596 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:30.507485 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cae0ebb4-337e-4a10-a754-f567691e1c6e-kube-api-access-nk76f" (OuterVolumeSpecName: "kube-api-access-nk76f") pod "cae0ebb4-337e-4a10-a754-f567691e1c6e" (UID: "cae0ebb4-337e-4a10-a754-f567691e1c6e"). InnerVolumeSpecName "kube-api-access-nk76f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:45:30.606818 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:30.606721 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nk76f\" (UniqueName: \"kubernetes.io/projected/cae0ebb4-337e-4a10-a754-f567691e1c6e-kube-api-access-nk76f\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:45:30.606818 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:30.606764 2565 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cae0ebb4-337e-4a10-a754-f567691e1c6e-webhook-cert\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 04:45:30.862252 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:30.862167 2565 generic.go:356] "Generic (PLEG): container finished" podID="cae0ebb4-337e-4a10-a754-f567691e1c6e" containerID="b5b7bfeb045ebc39c1558e15572369bf11387627fd93bfe77266ddad353d9d9f" exitCode=0 Jul 09 04:45:30.862252 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:30.862207 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d78b58cc-bxrcx" event={"ID":"cae0ebb4-337e-4a10-a754-f567691e1c6e","Type":"ContainerDied","Data":"b5b7bfeb045ebc39c1558e15572369bf11387627fd93bfe77266ddad353d9d9f"} Jul 09 04:45:30.862252 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:30.862246 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d78b58cc-bxrcx" event={"ID":"cae0ebb4-337e-4a10-a754-f567691e1c6e","Type":"ContainerDied","Data":"7856ccbfd706f77bdb2ddb3925d7e2b07d8f69e6dfc318fe52975dd0a63d9810"} Jul 09 04:45:30.862510 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:30.862267 2565 scope.go:117] "RemoveContainer" containerID="b5b7bfeb045ebc39c1558e15572369bf11387627fd93bfe77266ddad353d9d9f" Jul 09 04:45:30.862510 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:30.862211 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d78b58cc-bxrcx" Jul 09 04:45:30.874736 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:30.874715 2565 scope.go:117] "RemoveContainer" containerID="b5b7bfeb045ebc39c1558e15572369bf11387627fd93bfe77266ddad353d9d9f" Jul 09 04:45:30.875039 ip-10-0-139-184 kubenswrapper[2565]: E0709 04:45:30.875018 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5b7bfeb045ebc39c1558e15572369bf11387627fd93bfe77266ddad353d9d9f\": container with ID starting with b5b7bfeb045ebc39c1558e15572369bf11387627fd93bfe77266ddad353d9d9f not found: ID does not exist" containerID="b5b7bfeb045ebc39c1558e15572369bf11387627fd93bfe77266ddad353d9d9f" Jul 09 04:45:30.875096 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:30.875048 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5b7bfeb045ebc39c1558e15572369bf11387627fd93bfe77266ddad353d9d9f"} err="failed to get container status \"b5b7bfeb045ebc39c1558e15572369bf11387627fd93bfe77266ddad353d9d9f\": rpc error: code = NotFound desc = could not find container \"b5b7bfeb045ebc39c1558e15572369bf11387627fd93bfe77266ddad353d9d9f\": container with ID starting with b5b7bfeb045ebc39c1558e15572369bf11387627fd93bfe77266ddad353d9d9f not found: ID does not exist" Jul 09 04:45:30.884508 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:30.884484 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d78b58cc-bxrcx"] Jul 09 04:45:30.887624 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:30.887597 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-d78b58cc-bxrcx"] Jul 09 04:45:31.513023 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:31.512989 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:45:31.615698 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:31.615662 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-d78b58cc-xvgdd"] Jul 09 04:45:31.616110 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:31.616092 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cae0ebb4-337e-4a10-a754-f567691e1c6e" containerName="manager" Jul 09 04:45:31.616258 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:31.616113 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="cae0ebb4-337e-4a10-a754-f567691e1c6e" containerName="manager" Jul 09 04:45:31.616258 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:31.616147 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd0e0e40-a52a-449d-b358-cabef23533e2" containerName="authorino" Jul 09 04:45:31.616258 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:31.616156 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd0e0e40-a52a-449d-b358-cabef23533e2" containerName="authorino" Jul 09 04:45:31.616449 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:31.616264 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd0e0e40-a52a-449d-b358-cabef23533e2" containerName="authorino" Jul 09 04:45:31.616449 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:31.616276 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="cae0ebb4-337e-4a10-a754-f567691e1c6e" containerName="manager" Jul 09 04:45:31.620594 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:31.620572 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d78b58cc-xvgdd" Jul 09 04:45:31.624110 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:31.623970 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-jxlrk\"" Jul 09 04:45:31.624110 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:31.623981 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 09 04:45:31.624110 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:31.624044 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 09 04:45:31.627048 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:31.627023 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d78b58cc-xvgdd"] Jul 09 04:45:31.716245 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:31.716206 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78pfq\" (UniqueName: \"kubernetes.io/projected/dd160605-cb97-41bb-8283-f5545fd734f6-kube-api-access-78pfq\") pod \"maas-controller-d78b58cc-xvgdd\" (UID: \"dd160605-cb97-41bb-8283-f5545fd734f6\") " pod="opendatahub/maas-controller-d78b58cc-xvgdd" Jul 09 04:45:31.716446 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:31.716381 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dd160605-cb97-41bb-8283-f5545fd734f6-webhook-cert\") pod \"maas-controller-d78b58cc-xvgdd\" (UID: \"dd160605-cb97-41bb-8283-f5545fd734f6\") " pod="opendatahub/maas-controller-d78b58cc-xvgdd" Jul 09 04:45:31.816919 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:31.816832 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dd160605-cb97-41bb-8283-f5545fd734f6-webhook-cert\") pod \"maas-controller-d78b58cc-xvgdd\" (UID: \"dd160605-cb97-41bb-8283-f5545fd734f6\") " pod="opendatahub/maas-controller-d78b58cc-xvgdd" Jul 09 04:45:31.816919 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:31.816877 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78pfq\" (UniqueName: \"kubernetes.io/projected/dd160605-cb97-41bb-8283-f5545fd734f6-kube-api-access-78pfq\") pod \"maas-controller-d78b58cc-xvgdd\" (UID: \"dd160605-cb97-41bb-8283-f5545fd734f6\") " pod="opendatahub/maas-controller-d78b58cc-xvgdd" Jul 09 04:45:31.819255 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:31.819233 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dd160605-cb97-41bb-8283-f5545fd734f6-webhook-cert\") pod \"maas-controller-d78b58cc-xvgdd\" (UID: \"dd160605-cb97-41bb-8283-f5545fd734f6\") " pod="opendatahub/maas-controller-d78b58cc-xvgdd" Jul 09 04:45:31.825372 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:31.825321 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-78pfq\" (UniqueName: \"kubernetes.io/projected/dd160605-cb97-41bb-8283-f5545fd734f6-kube-api-access-78pfq\") pod \"maas-controller-d78b58cc-xvgdd\" (UID: \"dd160605-cb97-41bb-8283-f5545fd734f6\") " pod="opendatahub/maas-controller-d78b58cc-xvgdd" Jul 09 04:45:31.933278 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:31.933241 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d78b58cc-xvgdd" Jul 09 04:45:32.072016 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:32.071982 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d78b58cc-xvgdd"] Jul 09 04:45:32.073881 ip-10-0-139-184 kubenswrapper[2565]: W0709 04:45:32.073852 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd160605_cb97_41bb_8283_f5545fd734f6.slice/crio-8f6bab2d03c6f2c01f827856b7bd3b753f3d2b5e33abce8b9adb9182433b1a25 WatchSource:0}: Error finding container 8f6bab2d03c6f2c01f827856b7bd3b753f3d2b5e33abce8b9adb9182433b1a25: Status 404 returned error can't find the container with id 8f6bab2d03c6f2c01f827856b7bd3b753f3d2b5e33abce8b9adb9182433b1a25 Jul 09 04:45:32.195946 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:32.195910 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cae0ebb4-337e-4a10-a754-f567691e1c6e" path="/var/lib/kubelet/pods/cae0ebb4-337e-4a10-a754-f567691e1c6e/volumes" Jul 09 04:45:32.874569 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:32.874533 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d78b58cc-xvgdd" event={"ID":"dd160605-cb97-41bb-8283-f5545fd734f6","Type":"ContainerStarted","Data":"aed1cd93f05ec614362f6e4d6e46378b3e5885c5629adb4bc1448769e62d622e"} Jul 09 04:45:32.874937 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:32.874661 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-d78b58cc-xvgdd" Jul 09 04:45:32.874937 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:32.874704 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d78b58cc-xvgdd" event={"ID":"dd160605-cb97-41bb-8283-f5545fd734f6","Type":"ContainerStarted","Data":"8f6bab2d03c6f2c01f827856b7bd3b753f3d2b5e33abce8b9adb9182433b1a25"} Jul 09 04:45:32.891148 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:32.891091 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-d78b58cc-xvgdd" podStartSLOduration=1.294521942 podStartE2EDuration="1.891076151s" totalImagesPullingTime="596.554209ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:45:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:45:32.889387617 +0000 UTC m=+805.140911195" watchObservedRunningTime="2026-07-09 04:45:32.891076151 +0000 UTC m=+805.142599728" Jul 09 04:45:43.491236 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:43.491166 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:45:43.555494 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:43.555460 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:45:43.886506 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:45:43.886412 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-d78b58cc-xvgdd" Jul 09 04:46:11.961032 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:46:11.960998 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:46:12.062329 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:46:12.062295 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:46:12.175114 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:46:12.175079 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:46:17.775619 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:46:17.775582 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:46:17.875909 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:46:17.875876 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:46:17.976501 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:46:17.976467 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:46:18.105641 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:46:18.105556 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:46:23.715426 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:46:23.715382 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:46:23.816294 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:46:23.816251 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:48:02.610691 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:48:02.610653 2565 generic.go:356] "Generic (PLEG): container finished" podID="3def7d50-d793-4f7b-bdd9-6c815bfe6b98" containerID="6dadbbc11d61db06ef2a3d72fd0a9a54b6ac3ba793ea9cd7577c06764c624736" exitCode=1 Jul 09 04:48:02.611163 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:48:02.610730 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p6wwc" event={"ID":"3def7d50-d793-4f7b-bdd9-6c815bfe6b98","Type":"ContainerDied","Data":"6dadbbc11d61db06ef2a3d72fd0a9a54b6ac3ba793ea9cd7577c06764c624736"} Jul 09 04:48:02.611163 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:48:02.611082 2565 scope.go:117] "RemoveContainer" containerID="6dadbbc11d61db06ef2a3d72fd0a9a54b6ac3ba793ea9cd7577c06764c624736" Jul 09 04:48:02.614190 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:48:02.614175 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 09 04:48:03.617252 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:48:03.617217 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p6wwc" event={"ID":"3def7d50-d793-4f7b-bdd9-6c815bfe6b98","Type":"ContainerStarted","Data":"f712913e3725a9a7de6d8c051e502c18ba37211ea634c80c1dce800c68c5f734"} Jul 09 04:48:03.617685 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:48:03.617425 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p6wwc" Jul 09 04:48:14.624013 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:48:14.623974 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p6wwc" Jul 09 04:49:33.156893 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:49:33.156859 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:49:34.213621 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:49:34.213586 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:50:06.601099 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:50:06.600999 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:50:06.654015 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:50:06.653973 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:54:42.381526 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:54:42.381446 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:55:52.317201 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:55:52.317166 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:56:23.805229 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:56:23.805189 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:56:23.856234 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:56:23.856193 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:57:36.728581 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:57:36.728542 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:57:44.098960 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:57:44.098915 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:57:52.349138 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:57:52.349095 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:57:57.408080 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:57:57.408049 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:00.108296 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:00.108253 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:00.207732 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:00.207689 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:00.561655 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:00.561622 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:06.190348 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:06.190304 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:06.294982 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:06.294937 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:06.396418 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:06.396375 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:06.465114 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:06.465032 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:07.501822 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:07.501789 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:07.599988 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:07.599937 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:07.783088 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:07.782994 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:07.882978 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:07.882936 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:08.316895 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:08.316864 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:08.417682 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:08.417648 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:09.861987 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:09.861956 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:12.572222 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:12.572187 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:12.766196 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:12.766170 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:12.864739 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:12.864654 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:12.972777 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:12.972746 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:14.638967 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:14.638922 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:18.786195 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:18.786163 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:18.886522 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:18.886483 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:19.000461 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:19.000421 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:19.101023 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:19.100934 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:20.097472 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:20.097395 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:20.207375 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:20.207327 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:20.379262 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:20.379135 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:20.531778 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:20.531746 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:20.832395 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:20.832349 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:20.932061 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:20.932026 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:22.180402 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:22.180362 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:25.182612 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:25.182575 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:25.507778 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:25.507667 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:31.234063 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:31.234027 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:31.334877 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:31.334814 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:32.689012 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:32.688980 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:37.037588 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:37.037504 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:37.137910 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:37.137878 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:37.245715 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:37.245675 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:37.346056 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:37.345972 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:38.329894 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:38.329861 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:38.407810 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:38.407782 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:38.472390 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:38.472331 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:38.890541 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:38.890511 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:38.991262 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:38.991227 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:39.235209 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:39.235180 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:40.311636 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:40.311606 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:42.769008 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:42.768949 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:42.983475 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:42.983437 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:43.055820 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:43.055710 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:43.257609 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:43.257581 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:44.705155 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:44.705123 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:48.897607 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:48.897566 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:48.995122 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:48.995084 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:49.359208 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:49.359171 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:49.423199 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:49.423154 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:49.490072 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:49.490021 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:50.872303 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:50.872269 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:56.576109 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:56.576073 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:56.648439 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:56.648398 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:56.720565 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:56.720527 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:56.821289 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:56.821258 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:58:57.244885 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:58:57.244853 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:59:23.197183 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:59:23.197144 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:59:28.937657 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:59:28.937620 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:59:29.004694 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:59:29.004647 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:59:29.093377 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:59:29.093305 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:59:29.195595 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:59:29.195508 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:59:45.014663 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:59:45.014623 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:59:50.815108 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:59:50.815074 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:59:50.873795 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:59:50.873762 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:59:50.951903 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:59:50.951871 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 04:59:51.051947 ip-10-0-139-184 kubenswrapper[2565]: I0709 04:59:51.051911 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:01.550796 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:01.550757 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:02.263737 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:02.263705 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:08.015513 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:08.015469 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:08.078911 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:08.078876 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:08.159117 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:08.159079 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:08.258885 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:08.258848 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:08.359897 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:08.359800 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:09.079665 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:09.079636 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:09.177013 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:09.176980 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:10.803282 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:10.803252 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:16.406916 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:16.406879 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:16.473452 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:16.473420 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:17.067209 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:17.067173 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:22.853327 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:22.853293 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:22.973486 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:22.973451 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:23.091560 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:23.091530 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:23.389776 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:23.389744 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:23.721071 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:23.721033 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:26.012352 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:26.012312 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:26.085725 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:26.085694 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:26.151821 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:26.151782 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:26.319503 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:26.319415 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:26.590869 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:26.590779 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:26.698902 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:26.698861 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:30.512211 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:30.512127 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:31.143160 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:31.143124 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:31.244547 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:31.244513 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:31.577425 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:31.577392 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:37.225996 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:37.225966 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:37.326609 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:37.326567 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:37.625108 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:37.625009 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:37.954427 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:37.954396 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:38.050436 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:38.050399 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:43.826168 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:43.826132 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:43.994659 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:43.994627 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:44.072833 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:44.072794 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:44.405943 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:44.405900 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:44.507498 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:44.507463 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:44.607123 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:44.607090 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:50.259048 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:50.259002 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:50.635620 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:50.635533 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:50.694139 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:50.694103 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:54.439162 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:54.439125 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:54.540165 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:54.540128 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:56.454384 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:56.454332 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:00:56.554523 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:00:56.554495 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:01:02.816481 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:02.816447 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:01:03.113928 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:03.113831 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:01:03.565000 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:03.564965 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:01:06.190795 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:06.190760 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:01:06.267362 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:06.267310 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:01:06.332131 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:06.332097 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:01:06.816986 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:06.816949 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:01:06.917719 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:06.917690 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:01:06.976055 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:06.976023 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:01:10.688531 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:10.688487 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:01:22.443691 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:22.443650 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6"] Jul 09 05:01:22.447524 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:22.447501 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6" Jul 09 05:01:22.449748 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:22.449727 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-8146f0d0\"/\"openshift-service-ca.crt\"" Jul 09 05:01:22.449881 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:22.449751 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-8146f0d0\"/\"default-dockercfg-r87r2\"" Jul 09 05:01:22.450823 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:22.450801 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-8146f0d0\"/\"e2e-lifecycle-model-8146f0d0-kserve-self-signed-certs\"" Jul 09 05:01:22.450823 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:22.450819 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-8146f0d0\"/\"kube-root-ca.crt\"" Jul 09 05:01:22.456176 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:22.456153 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6"] Jul 09 05:01:22.540148 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:22.540114 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:01:22.606032 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:22.606000 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07a2d3ea-4954-4b59-9825-498e6fb5e035-home\") pod \"e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6\" (UID: \"07a2d3ea-4954-4b59-9825-498e6fb5e035\") " pod="ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6" Jul 09 05:01:22.606032 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:22.606032 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07a2d3ea-4954-4b59-9825-498e6fb5e035-model-cache\") pod \"e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6\" (UID: \"07a2d3ea-4954-4b59-9825-498e6fb5e035\") " pod="ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6" Jul 09 05:01:22.606263 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:22.606059 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07a2d3ea-4954-4b59-9825-498e6fb5e035-tmp-dir\") pod \"e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6\" (UID: \"07a2d3ea-4954-4b59-9825-498e6fb5e035\") " pod="ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6" Jul 09 05:01:22.606263 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:22.606168 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07a2d3ea-4954-4b59-9825-498e6fb5e035-tls-certs\") pod \"e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6\" (UID: \"07a2d3ea-4954-4b59-9825-498e6fb5e035\") " pod="ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6" Jul 09 05:01:22.606263 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:22.606217 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07a2d3ea-4954-4b59-9825-498e6fb5e035-dshm\") pod \"e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6\" (UID: \"07a2d3ea-4954-4b59-9825-498e6fb5e035\") " pod="ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6" Jul 09 05:01:22.606263 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:22.606259 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxt4w\" (UniqueName: \"kubernetes.io/projected/07a2d3ea-4954-4b59-9825-498e6fb5e035-kube-api-access-gxt4w\") pod \"e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6\" (UID: \"07a2d3ea-4954-4b59-9825-498e6fb5e035\") " pod="ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6" Jul 09 05:01:22.707535 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:22.707439 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07a2d3ea-4954-4b59-9825-498e6fb5e035-dshm\") pod \"e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6\" (UID: \"07a2d3ea-4954-4b59-9825-498e6fb5e035\") " pod="ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6" Jul 09 05:01:22.707535 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:22.707479 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxt4w\" (UniqueName: \"kubernetes.io/projected/07a2d3ea-4954-4b59-9825-498e6fb5e035-kube-api-access-gxt4w\") pod \"e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6\" (UID: \"07a2d3ea-4954-4b59-9825-498e6fb5e035\") " pod="ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6" Jul 09 05:01:22.707766 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:22.707639 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07a2d3ea-4954-4b59-9825-498e6fb5e035-home\") pod \"e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6\" (UID: \"07a2d3ea-4954-4b59-9825-498e6fb5e035\") " pod="ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6" Jul 09 05:01:22.707766 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:22.707680 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07a2d3ea-4954-4b59-9825-498e6fb5e035-model-cache\") pod \"e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6\" (UID: \"07a2d3ea-4954-4b59-9825-498e6fb5e035\") " pod="ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6" Jul 09 05:01:22.707766 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:22.707732 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07a2d3ea-4954-4b59-9825-498e6fb5e035-tmp-dir\") pod \"e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6\" (UID: \"07a2d3ea-4954-4b59-9825-498e6fb5e035\") " pod="ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6" Jul 09 05:01:22.707928 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:22.707835 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07a2d3ea-4954-4b59-9825-498e6fb5e035-tls-certs\") pod \"e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6\" (UID: \"07a2d3ea-4954-4b59-9825-498e6fb5e035\") " pod="ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6" Jul 09 05:01:22.708061 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:22.708028 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07a2d3ea-4954-4b59-9825-498e6fb5e035-home\") pod \"e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6\" (UID: \"07a2d3ea-4954-4b59-9825-498e6fb5e035\") " pod="ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6" Jul 09 05:01:22.708125 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:22.708112 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07a2d3ea-4954-4b59-9825-498e6fb5e035-model-cache\") pod \"e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6\" (UID: \"07a2d3ea-4954-4b59-9825-498e6fb5e035\") " pod="ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6" Jul 09 05:01:22.708200 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:22.708180 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07a2d3ea-4954-4b59-9825-498e6fb5e035-tmp-dir\") pod \"e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6\" (UID: \"07a2d3ea-4954-4b59-9825-498e6fb5e035\") " pod="ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6" Jul 09 05:01:22.709844 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:22.709824 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07a2d3ea-4954-4b59-9825-498e6fb5e035-dshm\") pod \"e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6\" (UID: \"07a2d3ea-4954-4b59-9825-498e6fb5e035\") " pod="ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6" Jul 09 05:01:22.710351 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:22.710321 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07a2d3ea-4954-4b59-9825-498e6fb5e035-tls-certs\") pod \"e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6\" (UID: \"07a2d3ea-4954-4b59-9825-498e6fb5e035\") " pod="ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6" Jul 09 05:01:22.716394 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:22.716375 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxt4w\" (UniqueName: \"kubernetes.io/projected/07a2d3ea-4954-4b59-9825-498e6fb5e035-kube-api-access-gxt4w\") pod \"e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6\" (UID: \"07a2d3ea-4954-4b59-9825-498e6fb5e035\") " pod="ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6" Jul 09 05:01:22.756859 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:22.756815 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6" Jul 09 05:01:22.894896 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:22.894664 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6"] Jul 09 05:01:22.901047 ip-10-0-139-184 kubenswrapper[2565]: W0709 05:01:22.901014 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07a2d3ea_4954_4b59_9825_498e6fb5e035.slice/crio-74d7d1a112b733ce6b8cbcf09bf93ca1634ac6ad32a0987091487dc476f7249a WatchSource:0}: Error finding container 74d7d1a112b733ce6b8cbcf09bf93ca1634ac6ad32a0987091487dc476f7249a: Status 404 returned error can't find the container with id 74d7d1a112b733ce6b8cbcf09bf93ca1634ac6ad32a0987091487dc476f7249a Jul 09 05:01:23.613390 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:23.613333 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6" event={"ID":"07a2d3ea-4954-4b59-9825-498e6fb5e035","Type":"ContainerStarted","Data":"257836107dddaa135e56402a3b72c79efd3f413f93303ec2b37d2288da973657"} Jul 09 05:01:23.613780 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:23.613398 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6" event={"ID":"07a2d3ea-4954-4b59-9825-498e6fb5e035","Type":"ContainerStarted","Data":"74d7d1a112b733ce6b8cbcf09bf93ca1634ac6ad32a0987091487dc476f7249a"} Jul 09 05:01:23.630033 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:23.629984 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6" podStartSLOduration=1.629968573 podStartE2EDuration="1.629968573s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 05:01:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 05:01:23.627815479 +0000 UTC m=+1755.879339061" watchObservedRunningTime="2026-07-09 05:01:23.629968573 +0000 UTC m=+1755.881492151" Jul 09 05:01:32.757711 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:32.757672 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6" Jul 09 05:01:32.758134 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:32.757739 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6" Jul 09 05:01:32.770626 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:32.770600 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6" Jul 09 05:01:33.670114 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:33.670086 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6" Jul 09 05:01:48.164476 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:48.164441 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:01:48.596708 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:48.596619 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:01:54.277984 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:54.277950 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:01:59.262622 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:59.262589 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6"] Jul 09 05:01:59.357169 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:01:59.357126 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:02:14.283955 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:14.283891 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6" podUID="07a2d3ea-4954-4b59-9825-498e6fb5e035" containerName="main" containerID="cri-o://257836107dddaa135e56402a3b72c79efd3f413f93303ec2b37d2288da973657" gracePeriod=45 Jul 09 05:02:14.542631 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:14.542563 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6" Jul 09 05:02:14.686610 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:14.686574 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07a2d3ea-4954-4b59-9825-498e6fb5e035-home\") pod \"07a2d3ea-4954-4b59-9825-498e6fb5e035\" (UID: \"07a2d3ea-4954-4b59-9825-498e6fb5e035\") " Jul 09 05:02:14.686610 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:14.686608 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07a2d3ea-4954-4b59-9825-498e6fb5e035-tmp-dir\") pod \"07a2d3ea-4954-4b59-9825-498e6fb5e035\" (UID: \"07a2d3ea-4954-4b59-9825-498e6fb5e035\") " Jul 09 05:02:14.686858 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:14.686637 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07a2d3ea-4954-4b59-9825-498e6fb5e035-model-cache\") pod \"07a2d3ea-4954-4b59-9825-498e6fb5e035\" (UID: \"07a2d3ea-4954-4b59-9825-498e6fb5e035\") " Jul 09 05:02:14.686858 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:14.686672 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxt4w\" (UniqueName: \"kubernetes.io/projected/07a2d3ea-4954-4b59-9825-498e6fb5e035-kube-api-access-gxt4w\") pod \"07a2d3ea-4954-4b59-9825-498e6fb5e035\" (UID: \"07a2d3ea-4954-4b59-9825-498e6fb5e035\") " Jul 09 05:02:14.686858 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:14.686691 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07a2d3ea-4954-4b59-9825-498e6fb5e035-tls-certs\") pod \"07a2d3ea-4954-4b59-9825-498e6fb5e035\" (UID: \"07a2d3ea-4954-4b59-9825-498e6fb5e035\") " Jul 09 05:02:14.686858 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:14.686777 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07a2d3ea-4954-4b59-9825-498e6fb5e035-dshm\") pod \"07a2d3ea-4954-4b59-9825-498e6fb5e035\" (UID: \"07a2d3ea-4954-4b59-9825-498e6fb5e035\") " Jul 09 05:02:14.687039 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:14.686935 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07a2d3ea-4954-4b59-9825-498e6fb5e035-home" (OuterVolumeSpecName: "home") pod "07a2d3ea-4954-4b59-9825-498e6fb5e035" (UID: "07a2d3ea-4954-4b59-9825-498e6fb5e035"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 05:02:14.687039 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:14.686949 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07a2d3ea-4954-4b59-9825-498e6fb5e035-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "07a2d3ea-4954-4b59-9825-498e6fb5e035" (UID: "07a2d3ea-4954-4b59-9825-498e6fb5e035"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 05:02:14.687039 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:14.686965 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07a2d3ea-4954-4b59-9825-498e6fb5e035-model-cache" (OuterVolumeSpecName: "model-cache") pod "07a2d3ea-4954-4b59-9825-498e6fb5e035" (UID: "07a2d3ea-4954-4b59-9825-498e6fb5e035"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 05:02:14.688985 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:14.688955 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07a2d3ea-4954-4b59-9825-498e6fb5e035-dshm" (OuterVolumeSpecName: "dshm") pod "07a2d3ea-4954-4b59-9825-498e6fb5e035" (UID: "07a2d3ea-4954-4b59-9825-498e6fb5e035"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 05:02:14.689075 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:14.688998 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07a2d3ea-4954-4b59-9825-498e6fb5e035-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "07a2d3ea-4954-4b59-9825-498e6fb5e035" (UID: "07a2d3ea-4954-4b59-9825-498e6fb5e035"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 05:02:14.689075 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:14.689029 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07a2d3ea-4954-4b59-9825-498e6fb5e035-kube-api-access-gxt4w" (OuterVolumeSpecName: "kube-api-access-gxt4w") pod "07a2d3ea-4954-4b59-9825-498e6fb5e035" (UID: "07a2d3ea-4954-4b59-9825-498e6fb5e035"). InnerVolumeSpecName "kube-api-access-gxt4w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 05:02:14.787845 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:14.787811 2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07a2d3ea-4954-4b59-9825-498e6fb5e035-dshm\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 05:02:14.787845 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:14.787838 2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07a2d3ea-4954-4b59-9825-498e6fb5e035-home\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 05:02:14.787845 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:14.787847 2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07a2d3ea-4954-4b59-9825-498e6fb5e035-tmp-dir\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 05:02:14.787845 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:14.787858 2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07a2d3ea-4954-4b59-9825-498e6fb5e035-model-cache\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 05:02:14.788129 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:14.787866 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxt4w\" (UniqueName: \"kubernetes.io/projected/07a2d3ea-4954-4b59-9825-498e6fb5e035-kube-api-access-gxt4w\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 05:02:14.788129 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:14.787875 2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07a2d3ea-4954-4b59-9825-498e6fb5e035-tls-certs\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 05:02:14.870413 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:14.870294 2565 generic.go:356] "Generic (PLEG): container finished" podID="07a2d3ea-4954-4b59-9825-498e6fb5e035" containerID="257836107dddaa135e56402a3b72c79efd3f413f93303ec2b37d2288da973657" exitCode=0 Jul 09 05:02:14.870413 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:14.870334 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6" Jul 09 05:02:14.870413 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:14.870368 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6" event={"ID":"07a2d3ea-4954-4b59-9825-498e6fb5e035","Type":"ContainerDied","Data":"257836107dddaa135e56402a3b72c79efd3f413f93303ec2b37d2288da973657"} Jul 09 05:02:14.870413 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:14.870405 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6" event={"ID":"07a2d3ea-4954-4b59-9825-498e6fb5e035","Type":"ContainerDied","Data":"74d7d1a112b733ce6b8cbcf09bf93ca1634ac6ad32a0987091487dc476f7249a"} Jul 09 05:02:14.870712 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:14.870422 2565 scope.go:117] "RemoveContainer" containerID="257836107dddaa135e56402a3b72c79efd3f413f93303ec2b37d2288da973657" Jul 09 05:02:14.880162 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:14.880143 2565 scope.go:117] "RemoveContainer" containerID="257836107dddaa135e56402a3b72c79efd3f413f93303ec2b37d2288da973657" Jul 09 05:02:14.880453 ip-10-0-139-184 kubenswrapper[2565]: E0709 05:02:14.880433 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"257836107dddaa135e56402a3b72c79efd3f413f93303ec2b37d2288da973657\": container with ID starting with 257836107dddaa135e56402a3b72c79efd3f413f93303ec2b37d2288da973657 not found: ID does not exist" containerID="257836107dddaa135e56402a3b72c79efd3f413f93303ec2b37d2288da973657" Jul 09 05:02:14.880525 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:14.880462 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"257836107dddaa135e56402a3b72c79efd3f413f93303ec2b37d2288da973657"} err="failed to get container status \"257836107dddaa135e56402a3b72c79efd3f413f93303ec2b37d2288da973657\": rpc error: code = NotFound desc = could not find container \"257836107dddaa135e56402a3b72c79efd3f413f93303ec2b37d2288da973657\": container with ID starting with 257836107dddaa135e56402a3b72c79efd3f413f93303ec2b37d2288da973657 not found: ID does not exist" Jul 09 05:02:14.891554 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:14.891527 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6"] Jul 09 05:02:14.895283 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:14.895258 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-8146f0d0/e2e-lifecycle-model-8146f0d0-kserve-67454d6fbc-k9tz6"] Jul 09 05:02:16.195717 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:16.195687 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07a2d3ea-4954-4b59-9825-498e6fb5e035" path="/var/lib/kubelet/pods/07a2d3ea-4954-4b59-9825-498e6fb5e035/volumes" Jul 09 05:02:35.640136 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:35.640091 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:02:36.265443 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:36.265411 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:02:42.131514 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:42.131477 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:02:42.231700 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:42.231665 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:02:42.418686 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:42.418660 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:02:43.713959 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:43.713906 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:02:43.819724 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:43.819687 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:02:43.913241 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:43.913204 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:02:44.259254 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:44.259218 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:02:44.311608 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:44.311576 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:02:44.664943 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:44.664909 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:02:44.761594 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:44.761558 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:02:44.862229 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:44.862200 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:02:46.259590 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:46.259558 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:02:51.905271 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:51.905237 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:02:52.009140 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:52.009106 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:02:52.564723 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:52.564692 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:02:58.348769 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:58.348734 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:02:58.578982 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:58.578949 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:02:58.680479 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:02:58.680445 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:03:09.031654 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:09.031614 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:03:09.370164 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:09.370073 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:03:21.093871 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:21.093834 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:03:21.194164 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:21.194126 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:03:30.320195 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:30.320113 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:03:36.047394 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:36.047324 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:03:36.146943 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:36.146908 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:03:36.382707 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:36.382619 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:03:36.668594 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:36.668555 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:03:37.021514 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:37.021426 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:03:39.904169 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:39.904133 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:03:39.978048 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:39.978017 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:03:40.043167 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:40.043139 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:03:40.262226 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:40.262196 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:03:40.587987 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:40.587879 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:03:40.688216 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:40.688187 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:03:44.418483 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:44.418452 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:03:44.729674 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:44.729588 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:03:44.990861 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:44.990781 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl"] Jul 09 05:03:44.991197 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:44.991183 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07a2d3ea-4954-4b59-9825-498e6fb5e035" containerName="main" Jul 09 05:03:44.991245 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:44.991198 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="07a2d3ea-4954-4b59-9825-498e6fb5e035" containerName="main" Jul 09 05:03:44.991280 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:44.991255 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="07a2d3ea-4954-4b59-9825-498e6fb5e035" containerName="main" Jul 09 05:03:44.994310 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:44.994289 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:03:44.997606 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:44.997579 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-64ed88-openshift-default-dockercfg-ls9gg\"" Jul 09 05:03:45.006333 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:45.006304 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl"] Jul 09 05:03:45.016116 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:45.016091 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:03:45.026565 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:45.026536 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7139bc3b-674c-4552-b0ff-6c890bd9d709-istio-data\") pod \"e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl\" (UID: \"7139bc3b-674c-4552-b0ff-6c890bd9d709\") " pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:03:45.026723 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:45.026573 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7139bc3b-674c-4552-b0ff-6c890bd9d709-istio-token\") pod \"e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl\" (UID: \"7139bc3b-674c-4552-b0ff-6c890bd9d709\") " pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:03:45.026723 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:45.026640 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bx64z\" (UniqueName: \"kubernetes.io/projected/7139bc3b-674c-4552-b0ff-6c890bd9d709-kube-api-access-bx64z\") pod \"e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl\" (UID: \"7139bc3b-674c-4552-b0ff-6c890bd9d709\") " pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:03:45.026723 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:45.026700 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7139bc3b-674c-4552-b0ff-6c890bd9d709-credential-socket\") pod \"e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl\" (UID: \"7139bc3b-674c-4552-b0ff-6c890bd9d709\") " pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:03:45.027087 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:45.026757 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7139bc3b-674c-4552-b0ff-6c890bd9d709-istio-podinfo\") pod \"e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl\" (UID: \"7139bc3b-674c-4552-b0ff-6c890bd9d709\") " pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:03:45.027087 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:45.026937 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7139bc3b-674c-4552-b0ff-6c890bd9d709-workload-certs\") pod \"e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl\" (UID: \"7139bc3b-674c-4552-b0ff-6c890bd9d709\") " pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:03:45.027087 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:45.026988 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7139bc3b-674c-4552-b0ff-6c890bd9d709-workload-socket\") pod \"e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl\" (UID: \"7139bc3b-674c-4552-b0ff-6c890bd9d709\") " pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:03:45.027087 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:45.027055 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7139bc3b-674c-4552-b0ff-6c890bd9d709-istio-envoy\") pod \"e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl\" (UID: \"7139bc3b-674c-4552-b0ff-6c890bd9d709\") " pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:03:45.027258 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:45.027124 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7139bc3b-674c-4552-b0ff-6c890bd9d709-istiod-ca-cert\") pod \"e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl\" (UID: \"7139bc3b-674c-4552-b0ff-6c890bd9d709\") " pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:03:45.127724 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:45.127689 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bx64z\" (UniqueName: \"kubernetes.io/projected/7139bc3b-674c-4552-b0ff-6c890bd9d709-kube-api-access-bx64z\") pod \"e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl\" (UID: \"7139bc3b-674c-4552-b0ff-6c890bd9d709\") " pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:03:45.127930 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:45.127737 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7139bc3b-674c-4552-b0ff-6c890bd9d709-credential-socket\") pod \"e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl\" (UID: \"7139bc3b-674c-4552-b0ff-6c890bd9d709\") " pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:03:45.127930 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:45.127756 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7139bc3b-674c-4552-b0ff-6c890bd9d709-istio-podinfo\") pod \"e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl\" (UID: \"7139bc3b-674c-4552-b0ff-6c890bd9d709\") " pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:03:45.127930 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:45.127779 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7139bc3b-674c-4552-b0ff-6c890bd9d709-workload-certs\") pod \"e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl\" (UID: \"7139bc3b-674c-4552-b0ff-6c890bd9d709\") " pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:03:45.127930 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:45.127803 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7139bc3b-674c-4552-b0ff-6c890bd9d709-workload-socket\") pod \"e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl\" (UID: \"7139bc3b-674c-4552-b0ff-6c890bd9d709\") " pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:03:45.127930 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:45.127818 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7139bc3b-674c-4552-b0ff-6c890bd9d709-istio-envoy\") pod \"e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl\" (UID: \"7139bc3b-674c-4552-b0ff-6c890bd9d709\") " pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:03:45.127930 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:45.127847 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7139bc3b-674c-4552-b0ff-6c890bd9d709-istiod-ca-cert\") pod \"e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl\" (UID: \"7139bc3b-674c-4552-b0ff-6c890bd9d709\") " pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:03:45.127930 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:45.127922 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7139bc3b-674c-4552-b0ff-6c890bd9d709-istio-data\") pod \"e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl\" (UID: \"7139bc3b-674c-4552-b0ff-6c890bd9d709\") " pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:03:45.128273 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:45.127946 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7139bc3b-674c-4552-b0ff-6c890bd9d709-istio-token\") pod \"e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl\" (UID: \"7139bc3b-674c-4552-b0ff-6c890bd9d709\") " pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:03:45.128273 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:45.128247 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7139bc3b-674c-4552-b0ff-6c890bd9d709-workload-certs\") pod \"e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl\" (UID: \"7139bc3b-674c-4552-b0ff-6c890bd9d709\") " pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:03:45.128406 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:45.128375 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7139bc3b-674c-4552-b0ff-6c890bd9d709-credential-socket\") pod \"e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl\" (UID: \"7139bc3b-674c-4552-b0ff-6c890bd9d709\") " pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:03:45.128462 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:45.128378 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7139bc3b-674c-4552-b0ff-6c890bd9d709-workload-socket\") pod \"e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl\" (UID: \"7139bc3b-674c-4552-b0ff-6c890bd9d709\") " pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:03:45.128517 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:45.128487 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7139bc3b-674c-4552-b0ff-6c890bd9d709-istio-data\") pod \"e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl\" (UID: \"7139bc3b-674c-4552-b0ff-6c890bd9d709\") " pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:03:45.128745 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:45.128724 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7139bc3b-674c-4552-b0ff-6c890bd9d709-istiod-ca-cert\") pod \"e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl\" (UID: \"7139bc3b-674c-4552-b0ff-6c890bd9d709\") " pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:03:45.130216 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:45.130192 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7139bc3b-674c-4552-b0ff-6c890bd9d709-istio-envoy\") pod \"e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl\" (UID: \"7139bc3b-674c-4552-b0ff-6c890bd9d709\") " pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:03:45.130323 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:45.130294 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7139bc3b-674c-4552-b0ff-6c890bd9d709-istio-podinfo\") pod \"e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl\" (UID: \"7139bc3b-674c-4552-b0ff-6c890bd9d709\") " pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:03:45.135688 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:45.135657 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7139bc3b-674c-4552-b0ff-6c890bd9d709-istio-token\") pod \"e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl\" (UID: \"7139bc3b-674c-4552-b0ff-6c890bd9d709\") " pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:03:45.135786 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:45.135771 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bx64z\" (UniqueName: \"kubernetes.io/projected/7139bc3b-674c-4552-b0ff-6c890bd9d709-kube-api-access-bx64z\") pod \"e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl\" (UID: \"7139bc3b-674c-4552-b0ff-6c890bd9d709\") " pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:03:45.251525 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:45.251442 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:03:45.308313 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:45.308281 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:03:45.449203 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:45.449179 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl"] Jul 09 05:03:45.451055 ip-10-0-139-184 kubenswrapper[2565]: W0709 05:03:45.451013 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7139bc3b_674c_4552_b0ff_6c890bd9d709.slice/crio-6217d76e5a542fcf8e89277ca9971b22e9a4fffc103f92dbc00937183787abb7 WatchSource:0}: Error finding container 6217d76e5a542fcf8e89277ca9971b22e9a4fffc103f92dbc00937183787abb7: Status 404 returned error can't find the container with id 6217d76e5a542fcf8e89277ca9971b22e9a4fffc103f92dbc00937183787abb7 Jul 09 05:03:45.453305 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:45.453261 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 05:03:45.453434 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:45.453380 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 05:03:45.453497 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:45.453432 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 05:03:46.321017 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:46.320975 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" event={"ID":"7139bc3b-674c-4552-b0ff-6c890bd9d709","Type":"ContainerStarted","Data":"f5d65d0616fe0f833ea0e5354166711d1b57dee92231c7db35dc8f3febcd4cbe"} Jul 09 05:03:46.321214 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:46.321041 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" event={"ID":"7139bc3b-674c-4552-b0ff-6c890bd9d709","Type":"ContainerStarted","Data":"6217d76e5a542fcf8e89277ca9971b22e9a4fffc103f92dbc00937183787abb7"} Jul 09 05:03:46.344541 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:46.344488 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" podStartSLOduration=2.3444710730000002 podStartE2EDuration="2.344471073s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 05:03:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 05:03:46.342935886 +0000 UTC m=+1898.594459463" watchObservedRunningTime="2026-07-09 05:03:46.344471073 +0000 UTC m=+1898.595994650" Jul 09 05:03:47.308767 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:47.308730 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:03:47.313637 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:47.313611 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:03:47.325665 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:47.325638 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:03:47.326437 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:47.326418 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:03:48.294540 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:48.294511 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:03:48.361404 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:48.361370 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:03:48.549944 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:48.549861 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:03:48.650176 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:48.650143 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:03:49.025243 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:49.025213 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:03:49.139528 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:49.139489 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:03:52.682934 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:52.682902 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:03:59.385176 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:59.385079 2565 generic.go:356] "Generic (PLEG): container finished" podID="7139bc3b-674c-4552-b0ff-6c890bd9d709" containerID="f5d65d0616fe0f833ea0e5354166711d1b57dee92231c7db35dc8f3febcd4cbe" exitCode=0 Jul 09 05:03:59.385176 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:59.385153 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" event={"ID":"7139bc3b-674c-4552-b0ff-6c890bd9d709","Type":"ContainerDied","Data":"f5d65d0616fe0f833ea0e5354166711d1b57dee92231c7db35dc8f3febcd4cbe"} Jul 09 05:03:59.385648 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:59.385567 2565 scope.go:117] "RemoveContainer" containerID="f5d65d0616fe0f833ea0e5354166711d1b57dee92231c7db35dc8f3febcd4cbe" Jul 09 05:03:59.386546 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:59.386519 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 05:03:59.386617 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:59.386604 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 05:03:59.386662 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:03:59.386639 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 05:04:00.391890 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:04:00.391850 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" event={"ID":"7139bc3b-674c-4552-b0ff-6c890bd9d709","Type":"ContainerStarted","Data":"9f0cf42432749050507d551e47aa45cccaba803adb378450fafe397639f7c61d"} Jul 09 05:04:01.305295 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:04:01.305251 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:04:01.310709 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:04:01.310669 2565 patch_prober.go:28] interesting pod/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 09 05:04:01.310881 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:04:01.310748 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" podUID="7139bc3b-674c-4552-b0ff-6c890bd9d709" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 09 05:04:01.311026 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:04:01.311004 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:04:01.406010 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:04:01.405967 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:04:02.310764 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:04:02.310727 2565 patch_prober.go:28] interesting pod/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 09 05:04:02.310976 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:04:02.310795 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" podUID="7139bc3b-674c-4552-b0ff-6c890bd9d709" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 09 05:04:02.326476 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:04:02.326438 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:04:03.309271 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:04:03.309219 2565 patch_prober.go:28] interesting pod/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 09 05:04:03.309833 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:04:03.309299 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" podUID="7139bc3b-674c-4552-b0ff-6c890bd9d709" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 09 05:04:04.313208 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:04:04.313173 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:04:04.314401 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:04:04.314382 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:04:05.183174 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:04:05.183138 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:04:10.547727 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:04:10.547693 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:04:10.646957 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:04:10.646922 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:04:41.871401 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:04:41.871365 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:05:06.860508 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:05:06.860433 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:05:08.257299 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:05:08.257266 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:05:08.470492 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:05:08.470451 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:05:08.570881 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:05:08.570790 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:05:08.671684 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:05:08.671645 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:05:08.798759 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:05:08.798715 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:05:13.548836 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:05:13.548795 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:05:50.207018 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:05:50.206979 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:05:50.472620 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:05:50.472539 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:05:50.859265 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:05:50.859184 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:05:51.261017 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:05:51.260987 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:05:51.360077 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:05:51.360044 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:05:51.461872 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:05:51.461840 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:05:51.561237 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:05:51.561151 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:05:55.123808 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:05:55.123764 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:05:55.223680 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:05:55.223634 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:05:57.492040 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:05:57.491986 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:05:57.524971 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:05:57.524939 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl"] Jul 09 05:05:57.525180 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:05:57.525146 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" podUID="7139bc3b-674c-4552-b0ff-6c890bd9d709" containerName="istio-proxy" containerID="cri-o://9f0cf42432749050507d551e47aa45cccaba803adb378450fafe397639f7c61d" gracePeriod=30 Jul 09 05:05:57.606572 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:05:57.606542 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:05:58.058534 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:05:58.058505 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:05:58.351772 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:05:58.351688 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:05:58.610588 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:05:58.610508 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:01.674959 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:01.674914 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:01.771157 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:01.771118 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:02.186942 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:02.186904 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:02.287202 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:02.287163 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:02.387731 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:02.387699 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:02.795175 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:02.795098 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:06:02.845840 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:02.845803 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7139bc3b-674c-4552-b0ff-6c890bd9d709-istiod-ca-cert\") pod \"7139bc3b-674c-4552-b0ff-6c890bd9d709\" (UID: \"7139bc3b-674c-4552-b0ff-6c890bd9d709\") " Jul 09 05:06:02.846020 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:02.845855 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7139bc3b-674c-4552-b0ff-6c890bd9d709-istio-token\") pod \"7139bc3b-674c-4552-b0ff-6c890bd9d709\" (UID: \"7139bc3b-674c-4552-b0ff-6c890bd9d709\") " Jul 09 05:06:02.846020 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:02.845874 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7139bc3b-674c-4552-b0ff-6c890bd9d709-workload-socket\") pod \"7139bc3b-674c-4552-b0ff-6c890bd9d709\" (UID: \"7139bc3b-674c-4552-b0ff-6c890bd9d709\") " Jul 09 05:06:02.846020 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:02.845982 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7139bc3b-674c-4552-b0ff-6c890bd9d709-istio-podinfo\") pod \"7139bc3b-674c-4552-b0ff-6c890bd9d709\" (UID: \"7139bc3b-674c-4552-b0ff-6c890bd9d709\") " Jul 09 05:06:02.846150 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:02.846030 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7139bc3b-674c-4552-b0ff-6c890bd9d709-istio-envoy\") pod \"7139bc3b-674c-4552-b0ff-6c890bd9d709\" (UID: \"7139bc3b-674c-4552-b0ff-6c890bd9d709\") " Jul 09 05:06:02.846150 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:02.846092 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7139bc3b-674c-4552-b0ff-6c890bd9d709-istio-data\") pod \"7139bc3b-674c-4552-b0ff-6c890bd9d709\" (UID: \"7139bc3b-674c-4552-b0ff-6c890bd9d709\") " Jul 09 05:06:02.846150 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:02.846122 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bx64z\" (UniqueName: \"kubernetes.io/projected/7139bc3b-674c-4552-b0ff-6c890bd9d709-kube-api-access-bx64z\") pod \"7139bc3b-674c-4552-b0ff-6c890bd9d709\" (UID: \"7139bc3b-674c-4552-b0ff-6c890bd9d709\") " Jul 09 05:06:02.846150 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:02.846148 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7139bc3b-674c-4552-b0ff-6c890bd9d709-credential-socket\") pod \"7139bc3b-674c-4552-b0ff-6c890bd9d709\" (UID: \"7139bc3b-674c-4552-b0ff-6c890bd9d709\") " Jul 09 05:06:02.846382 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:02.846166 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7139bc3b-674c-4552-b0ff-6c890bd9d709-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7139bc3b-674c-4552-b0ff-6c890bd9d709" (UID: "7139bc3b-674c-4552-b0ff-6c890bd9d709"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 05:06:02.846382 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:02.846173 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7139bc3b-674c-4552-b0ff-6c890bd9d709-workload-certs\") pod \"7139bc3b-674c-4552-b0ff-6c890bd9d709\" (UID: \"7139bc3b-674c-4552-b0ff-6c890bd9d709\") " Jul 09 05:06:02.846382 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:02.846238 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7139bc3b-674c-4552-b0ff-6c890bd9d709-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7139bc3b-674c-4552-b0ff-6c890bd9d709" (UID: "7139bc3b-674c-4552-b0ff-6c890bd9d709"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 05:06:02.846578 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:02.846546 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7139bc3b-674c-4552-b0ff-6c890bd9d709-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7139bc3b-674c-4552-b0ff-6c890bd9d709" (UID: "7139bc3b-674c-4552-b0ff-6c890bd9d709"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 05:06:02.846578 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:02.846553 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7139bc3b-674c-4552-b0ff-6c890bd9d709-istiod-ca-cert\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 05:06:02.846705 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:02.846591 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7139bc3b-674c-4552-b0ff-6c890bd9d709-workload-socket\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 05:06:02.846767 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:02.846744 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7139bc3b-674c-4552-b0ff-6c890bd9d709-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7139bc3b-674c-4552-b0ff-6c890bd9d709" (UID: "7139bc3b-674c-4552-b0ff-6c890bd9d709"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 05:06:02.846988 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:02.846959 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7139bc3b-674c-4552-b0ff-6c890bd9d709-istio-data" (OuterVolumeSpecName: "istio-data") pod "7139bc3b-674c-4552-b0ff-6c890bd9d709" (UID: "7139bc3b-674c-4552-b0ff-6c890bd9d709"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 05:06:02.848254 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:02.848220 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7139bc3b-674c-4552-b0ff-6c890bd9d709-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7139bc3b-674c-4552-b0ff-6c890bd9d709" (UID: "7139bc3b-674c-4552-b0ff-6c890bd9d709"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 05:06:02.848390 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:02.848363 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7139bc3b-674c-4552-b0ff-6c890bd9d709-istio-token" (OuterVolumeSpecName: "istio-token") pod "7139bc3b-674c-4552-b0ff-6c890bd9d709" (UID: "7139bc3b-674c-4552-b0ff-6c890bd9d709"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 05:06:02.848627 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:02.848606 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7139bc3b-674c-4552-b0ff-6c890bd9d709-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7139bc3b-674c-4552-b0ff-6c890bd9d709" (UID: "7139bc3b-674c-4552-b0ff-6c890bd9d709"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 05:06:02.848691 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:02.848657 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7139bc3b-674c-4552-b0ff-6c890bd9d709-kube-api-access-bx64z" (OuterVolumeSpecName: "kube-api-access-bx64z") pod "7139bc3b-674c-4552-b0ff-6c890bd9d709" (UID: "7139bc3b-674c-4552-b0ff-6c890bd9d709"). InnerVolumeSpecName "kube-api-access-bx64z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 05:06:02.947253 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:02.947211 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bx64z\" (UniqueName: \"kubernetes.io/projected/7139bc3b-674c-4552-b0ff-6c890bd9d709-kube-api-access-bx64z\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 05:06:02.947253 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:02.947248 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7139bc3b-674c-4552-b0ff-6c890bd9d709-credential-socket\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 05:06:02.947253 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:02.947261 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7139bc3b-674c-4552-b0ff-6c890bd9d709-workload-certs\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 05:06:02.947542 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:02.947273 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7139bc3b-674c-4552-b0ff-6c890bd9d709-istio-token\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 05:06:02.947542 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:02.947284 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7139bc3b-674c-4552-b0ff-6c890bd9d709-istio-podinfo\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 05:06:02.947542 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:02.947294 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7139bc3b-674c-4552-b0ff-6c890bd9d709-istio-envoy\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 05:06:02.947542 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:02.947304 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7139bc3b-674c-4552-b0ff-6c890bd9d709-istio-data\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 05:06:03.011356 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:03.011316 2565 generic.go:356] "Generic (PLEG): container finished" podID="7139bc3b-674c-4552-b0ff-6c890bd9d709" containerID="9f0cf42432749050507d551e47aa45cccaba803adb378450fafe397639f7c61d" exitCode=0 Jul 09 05:06:03.011557 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:03.011393 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" Jul 09 05:06:03.011557 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:03.011401 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" event={"ID":"7139bc3b-674c-4552-b0ff-6c890bd9d709","Type":"ContainerDied","Data":"9f0cf42432749050507d551e47aa45cccaba803adb378450fafe397639f7c61d"} Jul 09 05:06:03.011557 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:03.011457 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl" event={"ID":"7139bc3b-674c-4552-b0ff-6c890bd9d709","Type":"ContainerDied","Data":"6217d76e5a542fcf8e89277ca9971b22e9a4fffc103f92dbc00937183787abb7"} Jul 09 05:06:03.011557 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:03.011482 2565 scope.go:117] "RemoveContainer" containerID="9f0cf42432749050507d551e47aa45cccaba803adb378450fafe397639f7c61d" Jul 09 05:06:03.020825 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:03.020803 2565 scope.go:117] "RemoveContainer" containerID="f5d65d0616fe0f833ea0e5354166711d1b57dee92231c7db35dc8f3febcd4cbe" Jul 09 05:06:03.029451 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:03.029432 2565 scope.go:117] "RemoveContainer" containerID="9f0cf42432749050507d551e47aa45cccaba803adb378450fafe397639f7c61d" Jul 09 05:06:03.029750 ip-10-0-139-184 kubenswrapper[2565]: E0709 05:06:03.029732 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f0cf42432749050507d551e47aa45cccaba803adb378450fafe397639f7c61d\": container with ID starting with 9f0cf42432749050507d551e47aa45cccaba803adb378450fafe397639f7c61d not found: ID does not exist" containerID="9f0cf42432749050507d551e47aa45cccaba803adb378450fafe397639f7c61d" Jul 09 05:06:03.029790 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:03.029760 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f0cf42432749050507d551e47aa45cccaba803adb378450fafe397639f7c61d"} err="failed to get container status \"9f0cf42432749050507d551e47aa45cccaba803adb378450fafe397639f7c61d\": rpc error: code = NotFound desc = could not find container \"9f0cf42432749050507d551e47aa45cccaba803adb378450fafe397639f7c61d\": container with ID starting with 9f0cf42432749050507d551e47aa45cccaba803adb378450fafe397639f7c61d not found: ID does not exist" Jul 09 05:06:03.029790 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:03.029777 2565 scope.go:117] "RemoveContainer" containerID="f5d65d0616fe0f833ea0e5354166711d1b57dee92231c7db35dc8f3febcd4cbe" Jul 09 05:06:03.030013 ip-10-0-139-184 kubenswrapper[2565]: E0709 05:06:03.029996 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5d65d0616fe0f833ea0e5354166711d1b57dee92231c7db35dc8f3febcd4cbe\": container with ID starting with f5d65d0616fe0f833ea0e5354166711d1b57dee92231c7db35dc8f3febcd4cbe not found: ID does not exist" containerID="f5d65d0616fe0f833ea0e5354166711d1b57dee92231c7db35dc8f3febcd4cbe" Jul 09 05:06:03.030051 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:03.030017 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5d65d0616fe0f833ea0e5354166711d1b57dee92231c7db35dc8f3febcd4cbe"} err="failed to get container status \"f5d65d0616fe0f833ea0e5354166711d1b57dee92231c7db35dc8f3febcd4cbe\": rpc error: code = NotFound desc = could not find container \"f5d65d0616fe0f833ea0e5354166711d1b57dee92231c7db35dc8f3febcd4cbe\": container with ID starting with f5d65d0616fe0f833ea0e5354166711d1b57dee92231c7db35dc8f3febcd4cbe not found: ID does not exist" Jul 09 05:06:03.050239 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:03.050158 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl"] Jul 09 05:06:03.062899 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:03.062866 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-64ed88-openshift-default-7db6479585-m42pl"] Jul 09 05:06:04.201072 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:04.201025 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7139bc3b-674c-4552-b0ff-6c890bd9d709" path="/var/lib/kubelet/pods/7139bc3b-674c-4552-b0ff-6c890bd9d709/volumes" Jul 09 05:06:06.134450 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.134411 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:06.507068 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.507026 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:06.773018 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.772939 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg"] Jul 09 05:06:06.773368 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.773329 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7139bc3b-674c-4552-b0ff-6c890bd9d709" containerName="istio-proxy" Jul 09 05:06:06.773368 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.773360 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7139bc3b-674c-4552-b0ff-6c890bd9d709" containerName="istio-proxy" Jul 09 05:06:06.773600 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.773465 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7139bc3b-674c-4552-b0ff-6c890bd9d709" containerName="istio-proxy" Jul 09 05:06:06.773600 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.773482 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7139bc3b-674c-4552-b0ff-6c890bd9d709" containerName="istio-proxy" Jul 09 05:06:06.777026 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.777005 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:06.779519 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.779496 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-9ad9dc-openshift-default-dockercfg-p8tg9\"" Jul 09 05:06:06.787697 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.787675 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg"] Jul 09 05:06:06.797747 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.797719 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:06.884331 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.884295 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-istiod-ca-cert\") pod \"e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg\" (UID: \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\") " pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:06.884539 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.884390 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-istio-token\") pod \"e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg\" (UID: \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\") " pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:06.884539 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.884494 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-workload-socket\") pod \"e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg\" (UID: \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\") " pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:06.884539 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.884525 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-istio-data\") pod \"e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg\" (UID: \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\") " pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:06.884681 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.884553 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-credential-socket\") pod \"e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg\" (UID: \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\") " pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:06.884681 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.884659 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-workload-certs\") pod \"e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg\" (UID: \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\") " pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:06.884773 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.884715 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lc4xh\" (UniqueName: \"kubernetes.io/projected/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-kube-api-access-lc4xh\") pod \"e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg\" (UID: \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\") " pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:06.884817 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.884788 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-istio-podinfo\") pod \"e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg\" (UID: \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\") " pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:06.884854 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.884844 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-istio-envoy\") pod \"e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg\" (UID: \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\") " pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:06.985922 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.985873 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-istio-token\") pod \"e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg\" (UID: \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\") " pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:06.986143 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.985940 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-workload-socket\") pod \"e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg\" (UID: \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\") " pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:06.986143 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.985967 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-istio-data\") pod \"e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg\" (UID: \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\") " pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:06.986143 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.986071 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-credential-socket\") pod \"e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg\" (UID: \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\") " pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:06.986143 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.986129 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-workload-certs\") pod \"e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg\" (UID: \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\") " pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:06.986401 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.986169 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lc4xh\" (UniqueName: \"kubernetes.io/projected/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-kube-api-access-lc4xh\") pod \"e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg\" (UID: \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\") " pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:06.986401 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.986223 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-istio-podinfo\") pod \"e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg\" (UID: \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\") " pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:06.986401 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.986279 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-istio-envoy\") pod \"e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg\" (UID: \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\") " pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:06.986401 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.986305 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-istiod-ca-cert\") pod \"e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg\" (UID: \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\") " pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:06.986401 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.986386 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-istio-data\") pod \"e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg\" (UID: \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\") " pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:06.986683 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.986415 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-workload-socket\") pod \"e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg\" (UID: \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\") " pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:06.986683 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.986591 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-credential-socket\") pod \"e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg\" (UID: \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\") " pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:06.986771 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.986693 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-workload-certs\") pod \"e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg\" (UID: \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\") " pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:06.987020 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.986996 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-istiod-ca-cert\") pod \"e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg\" (UID: \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\") " pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:06.988609 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.988579 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-istio-envoy\") pod \"e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg\" (UID: \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\") " pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:06.988838 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.988817 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-istio-podinfo\") pod \"e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg\" (UID: \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\") " pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:06.993669 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.993646 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lc4xh\" (UniqueName: \"kubernetes.io/projected/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-kube-api-access-lc4xh\") pod \"e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg\" (UID: \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\") " pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:06.994115 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:06.994097 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-istio-token\") pod \"e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg\" (UID: \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\") " pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:07.058052 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:07.057966 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:07.087855 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:07.087821 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:07.229537 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:07.229512 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg"] Jul 09 05:06:07.231514 ip-10-0-139-184 kubenswrapper[2565]: W0709 05:06:07.231487 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21c118d0_de2e_4e73_a6b6_4430f6a5ca44.slice/crio-e79b2268b05a59278602be7319e9fd4d359f2c83e9f03dd92fc979f3df60b4a6 WatchSource:0}: Error finding container e79b2268b05a59278602be7319e9fd4d359f2c83e9f03dd92fc979f3df60b4a6: Status 404 returned error can't find the container with id e79b2268b05a59278602be7319e9fd4d359f2c83e9f03dd92fc979f3df60b4a6 Jul 09 05:06:07.233852 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:07.233811 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 05:06:07.233936 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:07.233913 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 05:06:07.233980 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:07.233962 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 05:06:08.040216 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:08.040186 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" event={"ID":"21c118d0-de2e-4e73-a6b6-4430f6a5ca44","Type":"ContainerStarted","Data":"fc5d8e0f014fa1d7a4389fc0e8513688c0dff32bb3576e22ade21866da2ef95a"} Jul 09 05:06:08.040429 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:08.040235 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" event={"ID":"21c118d0-de2e-4e73-a6b6-4430f6a5ca44","Type":"ContainerStarted","Data":"e79b2268b05a59278602be7319e9fd4d359f2c83e9f03dd92fc979f3df60b4a6"} Jul 09 05:06:08.061143 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:08.061095 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" podStartSLOduration=2.06107731 podStartE2EDuration="2.06107731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 05:06:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 05:06:08.059283601 +0000 UTC m=+2040.310807179" watchObservedRunningTime="2026-07-09 05:06:08.06107731 +0000 UTC m=+2040.312601295" Jul 09 05:06:08.088681 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:08.088652 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:08.093696 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:08.093667 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:09.045727 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:09.045699 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:09.046892 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:09.046868 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:10.491544 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:10.491511 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:10.588193 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:10.588162 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:10.712478 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:10.712443 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:10.816109 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:10.816025 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:14.429065 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:14.429029 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:22.405942 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:22.405903 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:22.851577 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:22.851493 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:22.980252 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:22.980221 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:23.081020 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:23.080986 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:23.305998 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:23.305930 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:25.087545 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:25.087509 2565 patch_prober.go:28] interesting pod/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 09 05:06:25.087939 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:25.087573 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" podUID="21c118d0-de2e-4e73-a6b6-4430f6a5ca44" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 09 05:06:26.145613 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:26.145512 2565 generic.go:356] "Generic (PLEG): container finished" podID="21c118d0-de2e-4e73-a6b6-4430f6a5ca44" containerID="fc5d8e0f014fa1d7a4389fc0e8513688c0dff32bb3576e22ade21866da2ef95a" exitCode=0 Jul 09 05:06:26.145613 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:26.145587 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" event={"ID":"21c118d0-de2e-4e73-a6b6-4430f6a5ca44","Type":"ContainerDied","Data":"fc5d8e0f014fa1d7a4389fc0e8513688c0dff32bb3576e22ade21866da2ef95a"} Jul 09 05:06:26.146097 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:26.146077 2565 scope.go:117] "RemoveContainer" containerID="fc5d8e0f014fa1d7a4389fc0e8513688c0dff32bb3576e22ade21866da2ef95a" Jul 09 05:06:26.147095 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:26.147062 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 05:06:26.147164 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:26.147135 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 05:06:26.147204 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:26.147167 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 05:06:27.152821 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:27.152789 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" event={"ID":"21c118d0-de2e-4e73-a6b6-4430f6a5ca44","Type":"ContainerStarted","Data":"31b0df83ee9ff4b9e581d233412b5f70e7029d08ea3c3c16984a2e716e67ac70"} Jul 09 05:06:28.088681 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:28.088644 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:28.088936 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:28.088914 2565 patch_prober.go:28] interesting pod/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 09 05:06:28.088984 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:28.088964 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" podUID="21c118d0-de2e-4e73-a6b6-4430f6a5ca44" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 09 05:06:28.531054 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:28.530987 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:28.631146 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:28.631117 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:28.786178 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:28.786099 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:29.088952 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:29.088867 2565 patch_prober.go:28] interesting pod/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 09 05:06:29.088952 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:29.088933 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" podUID="21c118d0-de2e-4e73-a6b6-4430f6a5ca44" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 09 05:06:29.155532 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:29.155502 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:29.526282 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:29.526250 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:29.626788 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:29.626747 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:29.726983 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:29.726898 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:29.861899 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:29.861823 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:30.089050 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:30.089020 2565 patch_prober.go:28] interesting pod/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 09 05:06:30.089227 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:30.089076 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" podUID="21c118d0-de2e-4e73-a6b6-4430f6a5ca44" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 09 05:06:31.115799 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:31.115756 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:31.116240 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:31.116067 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:31.117143 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:31.117127 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:35.176248 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:35.176209 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:35.228037 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:35.228002 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:35.419881 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:35.419849 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg"] Jul 09 05:06:35.420069 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:35.420043 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" podUID="21c118d0-de2e-4e73-a6b6-4430f6a5ca44" containerName="istio-proxy" containerID="cri-o://31b0df83ee9ff4b9e581d233412b5f70e7029d08ea3c3c16984a2e716e67ac70" gracePeriod=30 Jul 09 05:06:35.438579 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:35.438488 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:35.971641 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:35.971601 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:36.262247 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:36.262167 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:36.389507 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:36.389475 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:38.759516 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:38.759428 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:38.851199 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:38.851166 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:38.919328 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:38.919294 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:39.019930 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:39.019844 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:39.425402 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:39.425364 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:39.552525 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:39.552493 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:39.907223 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:39.907192 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:40.689759 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:40.689690 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:40.719834 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:40.719798 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-istiod-ca-cert\") pod \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\" (UID: \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\") " Jul 09 05:06:40.719834 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:40.719831 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-istio-data\") pod \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\" (UID: \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\") " Jul 09 05:06:40.720054 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:40.719853 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lc4xh\" (UniqueName: \"kubernetes.io/projected/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-kube-api-access-lc4xh\") pod \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\" (UID: \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\") " Jul 09 05:06:40.720054 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:40.719880 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-workload-socket\") pod \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\" (UID: \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\") " Jul 09 05:06:40.720262 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:40.720217 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "21c118d0-de2e-4e73-a6b6-4430f6a5ca44" (UID: "21c118d0-de2e-4e73-a6b6-4430f6a5ca44"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 05:06:40.720354 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:40.720316 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "21c118d0-de2e-4e73-a6b6-4430f6a5ca44" (UID: "21c118d0-de2e-4e73-a6b6-4430f6a5ca44"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 05:06:40.720421 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:40.720356 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-workload-certs\") pod \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\" (UID: \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\") " Jul 09 05:06:40.720480 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:40.720445 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-istio-envoy\") pod \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\" (UID: \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\") " Jul 09 05:06:40.720549 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:40.720530 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "21c118d0-de2e-4e73-a6b6-4430f6a5ca44" (UID: "21c118d0-de2e-4e73-a6b6-4430f6a5ca44"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 05:06:40.720608 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:40.720594 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-credential-socket\") pod \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\" (UID: \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\") " Jul 09 05:06:40.720695 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:40.720667 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-istio-token\") pod \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\" (UID: \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\") " Jul 09 05:06:40.720768 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:40.720711 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-istio-podinfo\") pod \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\" (UID: \"21c118d0-de2e-4e73-a6b6-4430f6a5ca44\") " Jul 09 05:06:40.721208 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:40.720970 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-istio-data" (OuterVolumeSpecName: "istio-data") pod "21c118d0-de2e-4e73-a6b6-4430f6a5ca44" (UID: "21c118d0-de2e-4e73-a6b6-4430f6a5ca44"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 05:06:40.721208 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:40.720982 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "21c118d0-de2e-4e73-a6b6-4430f6a5ca44" (UID: "21c118d0-de2e-4e73-a6b6-4430f6a5ca44"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 05:06:40.721208 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:40.721045 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-istiod-ca-cert\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 05:06:40.721208 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:40.721065 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-workload-socket\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 05:06:40.721208 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:40.721080 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-workload-certs\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 05:06:40.722523 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:40.722494 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-kube-api-access-lc4xh" (OuterVolumeSpecName: "kube-api-access-lc4xh") pod "21c118d0-de2e-4e73-a6b6-4430f6a5ca44" (UID: "21c118d0-de2e-4e73-a6b6-4430f6a5ca44"). InnerVolumeSpecName "kube-api-access-lc4xh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 05:06:40.723010 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:40.722979 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "21c118d0-de2e-4e73-a6b6-4430f6a5ca44" (UID: "21c118d0-de2e-4e73-a6b6-4430f6a5ca44"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 05:06:40.723178 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:40.723159 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-istio-token" (OuterVolumeSpecName: "istio-token") pod "21c118d0-de2e-4e73-a6b6-4430f6a5ca44" (UID: "21c118d0-de2e-4e73-a6b6-4430f6a5ca44"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 05:06:40.723280 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:40.723259 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "21c118d0-de2e-4e73-a6b6-4430f6a5ca44" (UID: "21c118d0-de2e-4e73-a6b6-4430f6a5ca44"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 05:06:40.821576 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:40.821542 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-istio-token\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 05:06:40.821576 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:40.821571 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-istio-podinfo\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 05:06:40.821576 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:40.821580 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-istio-data\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 05:06:40.821821 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:40.821589 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lc4xh\" (UniqueName: \"kubernetes.io/projected/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-kube-api-access-lc4xh\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 05:06:40.821821 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:40.821599 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-istio-envoy\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 05:06:40.821821 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:40.821607 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21c118d0-de2e-4e73-a6b6-4430f6a5ca44-credential-socket\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 05:06:41.226840 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:41.226799 2565 generic.go:356] "Generic (PLEG): container finished" podID="21c118d0-de2e-4e73-a6b6-4430f6a5ca44" containerID="31b0df83ee9ff4b9e581d233412b5f70e7029d08ea3c3c16984a2e716e67ac70" exitCode=0 Jul 09 05:06:41.226840 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:41.226831 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" event={"ID":"21c118d0-de2e-4e73-a6b6-4430f6a5ca44","Type":"ContainerDied","Data":"31b0df83ee9ff4b9e581d233412b5f70e7029d08ea3c3c16984a2e716e67ac70"} Jul 09 05:06:41.227320 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:41.226868 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" Jul 09 05:06:41.227320 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:41.226885 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg" event={"ID":"21c118d0-de2e-4e73-a6b6-4430f6a5ca44","Type":"ContainerDied","Data":"e79b2268b05a59278602be7319e9fd4d359f2c83e9f03dd92fc979f3df60b4a6"} Jul 09 05:06:41.227320 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:41.226906 2565 scope.go:117] "RemoveContainer" containerID="31b0df83ee9ff4b9e581d233412b5f70e7029d08ea3c3c16984a2e716e67ac70" Jul 09 05:06:41.236372 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:41.236331 2565 scope.go:117] "RemoveContainer" containerID="fc5d8e0f014fa1d7a4389fc0e8513688c0dff32bb3576e22ade21866da2ef95a" Jul 09 05:06:41.244932 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:41.244910 2565 scope.go:117] "RemoveContainer" containerID="31b0df83ee9ff4b9e581d233412b5f70e7029d08ea3c3c16984a2e716e67ac70" Jul 09 05:06:41.245215 ip-10-0-139-184 kubenswrapper[2565]: E0709 05:06:41.245193 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31b0df83ee9ff4b9e581d233412b5f70e7029d08ea3c3c16984a2e716e67ac70\": container with ID starting with 31b0df83ee9ff4b9e581d233412b5f70e7029d08ea3c3c16984a2e716e67ac70 not found: ID does not exist" containerID="31b0df83ee9ff4b9e581d233412b5f70e7029d08ea3c3c16984a2e716e67ac70" Jul 09 05:06:41.245288 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:41.245230 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31b0df83ee9ff4b9e581d233412b5f70e7029d08ea3c3c16984a2e716e67ac70"} err="failed to get container status \"31b0df83ee9ff4b9e581d233412b5f70e7029d08ea3c3c16984a2e716e67ac70\": rpc error: code = NotFound desc = could not find container \"31b0df83ee9ff4b9e581d233412b5f70e7029d08ea3c3c16984a2e716e67ac70\": container with ID starting with 31b0df83ee9ff4b9e581d233412b5f70e7029d08ea3c3c16984a2e716e67ac70 not found: ID does not exist" Jul 09 05:06:41.245288 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:41.245258 2565 scope.go:117] "RemoveContainer" containerID="fc5d8e0f014fa1d7a4389fc0e8513688c0dff32bb3576e22ade21866da2ef95a" Jul 09 05:06:41.245546 ip-10-0-139-184 kubenswrapper[2565]: E0709 05:06:41.245527 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc5d8e0f014fa1d7a4389fc0e8513688c0dff32bb3576e22ade21866da2ef95a\": container with ID starting with fc5d8e0f014fa1d7a4389fc0e8513688c0dff32bb3576e22ade21866da2ef95a not found: ID does not exist" containerID="fc5d8e0f014fa1d7a4389fc0e8513688c0dff32bb3576e22ade21866da2ef95a" Jul 09 05:06:41.245608 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:41.245552 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc5d8e0f014fa1d7a4389fc0e8513688c0dff32bb3576e22ade21866da2ef95a"} err="failed to get container status \"fc5d8e0f014fa1d7a4389fc0e8513688c0dff32bb3576e22ade21866da2ef95a\": rpc error: code = NotFound desc = could not find container \"fc5d8e0f014fa1d7a4389fc0e8513688c0dff32bb3576e22ade21866da2ef95a\": container with ID starting with fc5d8e0f014fa1d7a4389fc0e8513688c0dff32bb3576e22ade21866da2ef95a not found: ID does not exist" Jul 09 05:06:41.250628 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:41.250605 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg"] Jul 09 05:06:41.253825 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:41.253804 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-9ad9dc-openshift-default-d8875d78f-wk8lg"] Jul 09 05:06:42.195904 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:42.195872 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21c118d0-de2e-4e73-a6b6-4430f6a5ca44" path="/var/lib/kubelet/pods/21c118d0-de2e-4e73-a6b6-4430f6a5ca44/volumes" Jul 09 05:06:43.244628 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:43.244589 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:43.557548 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:43.557453 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:43.821230 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:43.821144 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52"] Jul 09 05:06:43.821659 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:43.821638 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21c118d0-de2e-4e73-a6b6-4430f6a5ca44" containerName="istio-proxy" Jul 09 05:06:43.821783 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:43.821663 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="21c118d0-de2e-4e73-a6b6-4430f6a5ca44" containerName="istio-proxy" Jul 09 05:06:43.821783 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:43.821674 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21c118d0-de2e-4e73-a6b6-4430f6a5ca44" containerName="istio-proxy" Jul 09 05:06:43.821783 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:43.821680 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="21c118d0-de2e-4e73-a6b6-4430f6a5ca44" containerName="istio-proxy" Jul 09 05:06:43.821783 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:43.821695 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7139bc3b-674c-4552-b0ff-6c890bd9d709" containerName="istio-proxy" Jul 09 05:06:43.821783 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:43.821700 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7139bc3b-674c-4552-b0ff-6c890bd9d709" containerName="istio-proxy" Jul 09 05:06:43.822044 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:43.821789 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="21c118d0-de2e-4e73-a6b6-4430f6a5ca44" containerName="istio-proxy" Jul 09 05:06:43.822044 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:43.821803 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="21c118d0-de2e-4e73-a6b6-4430f6a5ca44" containerName="istio-proxy" Jul 09 05:06:43.824444 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:43.824425 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:06:43.827072 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:43.827053 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-33798d-openshift-default-dockercfg-l72sh\"" Jul 09 05:06:43.836191 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:43.836167 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52"] Jul 09 05:06:43.848549 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:43.848515 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:43.949222 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:43.949189 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-istio-podinfo\") pod \"e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52\" (UID: \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\") " pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:06:43.949222 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:43.949224 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-istio-data\") pod \"e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52\" (UID: \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\") " pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:06:43.949463 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:43.949288 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-workload-certs\") pod \"e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52\" (UID: \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\") " pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:06:43.949463 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:43.949304 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-istio-token\") pod \"e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52\" (UID: \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\") " pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:06:43.949463 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:43.949381 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-workload-socket\") pod \"e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52\" (UID: \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\") " pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:06:43.949562 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:43.949462 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-credential-socket\") pod \"e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52\" (UID: \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\") " pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:06:43.949562 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:43.949500 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmqtx\" (UniqueName: \"kubernetes.io/projected/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-kube-api-access-vmqtx\") pod \"e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52\" (UID: \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\") " pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:06:43.949562 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:43.949526 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-istio-envoy\") pod \"e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52\" (UID: \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\") " pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:06:43.949562 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:43.949544 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-istiod-ca-cert\") pod \"e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52\" (UID: \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\") " pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:06:44.050648 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:44.050612 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-workload-socket\") pod \"e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52\" (UID: \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\") " pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:06:44.050648 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:44.050664 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-credential-socket\") pod \"e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52\" (UID: \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\") " pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:06:44.050884 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:44.050686 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmqtx\" (UniqueName: \"kubernetes.io/projected/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-kube-api-access-vmqtx\") pod \"e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52\" (UID: \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\") " pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:06:44.050884 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:44.050717 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-istio-envoy\") pod \"e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52\" (UID: \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\") " pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:06:44.050884 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:44.050839 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-istiod-ca-cert\") pod \"e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52\" (UID: \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\") " pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:06:44.051040 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:44.050930 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-istio-podinfo\") pod \"e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52\" (UID: \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\") " pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:06:44.051040 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:44.050963 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-istio-data\") pod \"e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52\" (UID: \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\") " pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:06:44.051040 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:44.051027 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-workload-certs\") pod \"e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52\" (UID: \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\") " pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:06:44.051210 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:44.051061 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-istio-token\") pod \"e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52\" (UID: \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\") " pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:06:44.051210 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:44.051139 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-workload-socket\") pod \"e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52\" (UID: \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\") " pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:06:44.051210 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:44.051181 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-credential-socket\") pod \"e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52\" (UID: \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\") " pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:06:44.051449 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:44.051425 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-istio-data\") pod \"e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52\" (UID: \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\") " pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:06:44.051512 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:44.051489 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-workload-certs\") pod \"e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52\" (UID: \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\") " pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:06:44.051648 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:44.051628 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-istiod-ca-cert\") pod \"e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52\" (UID: \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\") " pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:06:44.053019 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:44.053002 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-istio-envoy\") pod \"e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52\" (UID: \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\") " pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:06:44.053312 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:44.053292 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-istio-podinfo\") pod \"e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52\" (UID: \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\") " pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:06:44.057903 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:44.057878 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-istio-token\") pod \"e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52\" (UID: \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\") " pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:06:44.058071 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:44.058052 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmqtx\" (UniqueName: \"kubernetes.io/projected/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-kube-api-access-vmqtx\") pod \"e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52\" (UID: \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\") " pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:06:44.122033 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:44.121931 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:44.135873 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:44.135844 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:06:44.276458 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:44.276433 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52"] Jul 09 05:06:44.278457 ip-10-0-139-184 kubenswrapper[2565]: W0709 05:06:44.278431 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b4b6fad_bb26_4132_a0aa_cfdab59e1b1a.slice/crio-02ccdc4fe7bcb3606f9dd992d3a90a34eca397fdf782f1275b83cbe6d5753821 WatchSource:0}: Error finding container 02ccdc4fe7bcb3606f9dd992d3a90a34eca397fdf782f1275b83cbe6d5753821: Status 404 returned error can't find the container with id 02ccdc4fe7bcb3606f9dd992d3a90a34eca397fdf782f1275b83cbe6d5753821 Jul 09 05:06:44.280594 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:44.280553 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 05:06:44.280683 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:44.280647 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 05:06:44.280729 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:44.280694 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 05:06:45.251639 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:45.251607 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" event={"ID":"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a","Type":"ContainerStarted","Data":"15a7fbe6006e93c464bbb14139fc42879f572f9555b2b3d8904ab7f1e35c0c2c"} Jul 09 05:06:45.251814 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:45.251660 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" event={"ID":"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a","Type":"ContainerStarted","Data":"02ccdc4fe7bcb3606f9dd992d3a90a34eca397fdf782f1275b83cbe6d5753821"} Jul 09 05:06:45.270386 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:45.270312 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" podStartSLOduration=2.270264562 podStartE2EDuration="2.270264562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 05:06:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 05:06:45.269516224 +0000 UTC m=+2077.521039808" watchObservedRunningTime="2026-07-09 05:06:45.270264562 +0000 UTC m=+2077.521788141" Jul 09 05:06:46.136593 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:46.136551 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:06:46.141421 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:46.141394 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:06:46.257376 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:46.257329 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:06:46.258400 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:46.258379 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:06:47.135493 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:47.135453 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:47.233569 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:47.233526 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:47.482750 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:47.482719 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:47.690422 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:47.690390 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:47.900794 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:47.900708 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:48.001707 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:48.001671 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:06:52.122929 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:06:52.122893 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:07:00.334405 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:07:00.334365 2565 generic.go:356] "Generic (PLEG): container finished" podID="4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a" containerID="15a7fbe6006e93c464bbb14139fc42879f572f9555b2b3d8904ab7f1e35c0c2c" exitCode=0 Jul 09 05:07:00.334820 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:07:00.334432 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" event={"ID":"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a","Type":"ContainerDied","Data":"15a7fbe6006e93c464bbb14139fc42879f572f9555b2b3d8904ab7f1e35c0c2c"} Jul 09 05:07:00.334820 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:07:00.334800 2565 scope.go:117] "RemoveContainer" containerID="15a7fbe6006e93c464bbb14139fc42879f572f9555b2b3d8904ab7f1e35c0c2c" Jul 09 05:07:00.335773 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:07:00.335745 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 05:07:00.335835 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:07:00.335806 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 05:07:00.335871 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:07:00.335838 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 05:07:01.090388 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:07:01.090324 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:07:01.189837 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:07:01.189800 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:07:01.257959 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:07:01.257928 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:07:01.341789 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:07:01.341700 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" event={"ID":"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a","Type":"ContainerStarted","Data":"713b28b43fc23de4c06c99541a4e9bda086e4ba8f52ec15dc06e652a9489ab88"} Jul 09 05:07:02.137028 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:07:02.136994 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:07:02.138183 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:07:02.138157 2565 patch_prober.go:28] interesting pod/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 09 05:07:02.138308 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:07:02.138221 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" podUID="4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 09 05:07:03.137060 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:07:03.137021 2565 patch_prober.go:28] interesting pod/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 09 05:07:03.137514 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:07:03.137078 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" podUID="4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 09 05:07:04.136381 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:07:04.136322 2565 patch_prober.go:28] interesting pod/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 09 05:07:04.136570 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:07:04.136409 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" podUID="4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 09 05:07:05.136789 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:07:05.136758 2565 patch_prober.go:28] interesting pod/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 09 05:07:05.137201 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:07:05.136812 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" podUID="4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 09 05:07:06.140356 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:07:06.140311 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:07:06.140755 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:07:06.140733 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:07:06.141433 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:07:06.141413 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:07:08.583400 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:07:08.583331 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:07:08.688200 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:07:08.688109 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:07:12.523578 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:07:12.523541 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:07:12.571283 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:07:12.571245 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:08:58.584713 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:08:58.584678 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:09:01.248412 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:09:01.248368 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:09:30.638081 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:09:30.637993 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:09:30.693079 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:09:30.693038 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:09:56.360446 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:09:56.360413 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:09:56.813242 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:09:56.813200 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:09:56.913656 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:09:56.913622 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:09:57.014985 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:09:57.014948 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:09:57.160542 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:09:57.160514 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:10:02.527574 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:02.527540 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:10:02.593154 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:02.593126 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:10:02.793224 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:02.793123 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:10:03.166969 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:03.166936 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:10:03.567435 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:03.567334 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:10:03.665325 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:03.665291 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:10:03.765834 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:03.765797 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:10:04.146083 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:04.146047 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:10:09.103652 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:09.103613 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:10:09.335485 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:09.335447 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52"] Jul 09 05:10:09.335781 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:09.335748 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" podUID="4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a" containerName="istio-proxy" containerID="cri-o://713b28b43fc23de4c06c99541a4e9bda086e4ba8f52ec15dc06e652a9489ab88" gracePeriod=30 Jul 09 05:10:10.101053 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:10.101018 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5b5d8996bc-hwqqk_e79be3a5-b853-4c59-95c6-7a914490b429/authorino/0.log" Jul 09 05:10:10.845690 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:10.845657 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:10:14.529852 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:14.529821 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-mx8xg_02072840-9f3b-43a1-a940-ac8aa8b1c188/manager/0.log" Jul 09 05:10:14.594251 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:14.594225 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:10:14.643493 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:14.643420 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-x8wrk_bed406fb-4a82-40d6-b190-5cf5033f6867/manager/0.log" Jul 09 05:10:14.654866 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:14.654841 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-workload-socket\") pod \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\" (UID: \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\") " Jul 09 05:10:14.655012 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:14.654871 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-credential-socket\") pod \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\" (UID: \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\") " Jul 09 05:10:14.655012 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:14.654894 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-istio-podinfo\") pod \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\" (UID: \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\") " Jul 09 05:10:14.655012 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:14.654913 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-istiod-ca-cert\") pod \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\" (UID: \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\") " Jul 09 05:10:14.655012 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:14.654935 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-istio-envoy\") pod \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\" (UID: \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\") " Jul 09 05:10:14.655012 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:14.654972 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-istio-token\") pod \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\" (UID: \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\") " Jul 09 05:10:14.655221 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:14.655027 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-istio-data\") pod \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\" (UID: \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\") " Jul 09 05:10:14.655221 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:14.655056 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-workload-certs\") pod \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\" (UID: \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\") " Jul 09 05:10:14.655221 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:14.655163 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a" (UID: "4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 05:10:14.655221 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:14.655196 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a" (UID: "4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 05:10:14.655385 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:14.655260 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vmqtx\" (UniqueName: \"kubernetes.io/projected/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-kube-api-access-vmqtx\") pod \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\" (UID: \"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a\") " Jul 09 05:10:14.655385 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:14.655273 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a" (UID: "4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 05:10:14.655661 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:14.655610 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a" (UID: "4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 05:10:14.655806 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:14.655785 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-workload-socket\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 05:10:14.655885 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:14.655810 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-credential-socket\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 05:10:14.655885 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:14.655825 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-istiod-ca-cert\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 05:10:14.655885 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:14.655842 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-workload-certs\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 05:10:14.656058 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:14.655984 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-istio-data" (OuterVolumeSpecName: "istio-data") pod "4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a" (UID: "4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 05:10:14.657297 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:14.657269 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-istio-token" (OuterVolumeSpecName: "istio-token") pod "4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a" (UID: "4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 05:10:14.657460 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:14.657293 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-kube-api-access-vmqtx" (OuterVolumeSpecName: "kube-api-access-vmqtx") pod "4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a" (UID: "4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a"). InnerVolumeSpecName "kube-api-access-vmqtx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 05:10:14.657460 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:14.657280 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a" (UID: "4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 05:10:14.657557 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:14.657460 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a" (UID: "4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 05:10:14.756542 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:14.756498 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-istio-data\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 05:10:14.756542 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:14.756547 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmqtx\" (UniqueName: \"kubernetes.io/projected/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-kube-api-access-vmqtx\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 05:10:14.756799 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:14.756559 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-istio-podinfo\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 05:10:14.756799 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:14.756576 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-istio-envoy\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 05:10:14.756799 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:14.756585 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a-istio-token\") on node \"ip-10-0-139-184.ec2.internal\" DevicePath \"\"" Jul 09 05:10:14.758456 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:14.758438 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-d78b58cc-xvgdd_dd160605-cb97-41bb-8283-f5545fd734f6/manager/0.log" Jul 09 05:10:14.864952 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:14.864925 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-9rgfm_91e3e24b-5366-4fea-a415-cd9e2bb46439/server/0.log" Jul 09 05:10:14.977506 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:14.977481 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-lzxrb_fb4b96ac-6cd9-4b45-a2b9-89667201bb54/manager/0.log" Jul 09 05:10:15.302658 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:15.302559 2565 generic.go:356] "Generic (PLEG): container finished" podID="4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a" containerID="713b28b43fc23de4c06c99541a4e9bda086e4ba8f52ec15dc06e652a9489ab88" exitCode=0 Jul 09 05:10:15.302658 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:15.302619 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" Jul 09 05:10:15.302658 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:15.302641 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" event={"ID":"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a","Type":"ContainerDied","Data":"713b28b43fc23de4c06c99541a4e9bda086e4ba8f52ec15dc06e652a9489ab88"} Jul 09 05:10:15.302929 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:15.302716 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52" event={"ID":"4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a","Type":"ContainerDied","Data":"02ccdc4fe7bcb3606f9dd992d3a90a34eca397fdf782f1275b83cbe6d5753821"} Jul 09 05:10:15.302929 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:15.302739 2565 scope.go:117] "RemoveContainer" containerID="713b28b43fc23de4c06c99541a4e9bda086e4ba8f52ec15dc06e652a9489ab88" Jul 09 05:10:15.312266 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:15.312248 2565 scope.go:117] "RemoveContainer" containerID="15a7fbe6006e93c464bbb14139fc42879f572f9555b2b3d8904ab7f1e35c0c2c" Jul 09 05:10:15.320240 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:15.320221 2565 scope.go:117] "RemoveContainer" containerID="713b28b43fc23de4c06c99541a4e9bda086e4ba8f52ec15dc06e652a9489ab88" Jul 09 05:10:15.320507 ip-10-0-139-184 kubenswrapper[2565]: E0709 05:10:15.320488 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"713b28b43fc23de4c06c99541a4e9bda086e4ba8f52ec15dc06e652a9489ab88\": container with ID starting with 713b28b43fc23de4c06c99541a4e9bda086e4ba8f52ec15dc06e652a9489ab88 not found: ID does not exist" containerID="713b28b43fc23de4c06c99541a4e9bda086e4ba8f52ec15dc06e652a9489ab88" Jul 09 05:10:15.320563 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:15.320516 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"713b28b43fc23de4c06c99541a4e9bda086e4ba8f52ec15dc06e652a9489ab88"} err="failed to get container status \"713b28b43fc23de4c06c99541a4e9bda086e4ba8f52ec15dc06e652a9489ab88\": rpc error: code = NotFound desc = could not find container \"713b28b43fc23de4c06c99541a4e9bda086e4ba8f52ec15dc06e652a9489ab88\": container with ID starting with 713b28b43fc23de4c06c99541a4e9bda086e4ba8f52ec15dc06e652a9489ab88 not found: ID does not exist" Jul 09 05:10:15.320563 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:15.320533 2565 scope.go:117] "RemoveContainer" containerID="15a7fbe6006e93c464bbb14139fc42879f572f9555b2b3d8904ab7f1e35c0c2c" Jul 09 05:10:15.320797 ip-10-0-139-184 kubenswrapper[2565]: E0709 05:10:15.320781 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15a7fbe6006e93c464bbb14139fc42879f572f9555b2b3d8904ab7f1e35c0c2c\": container with ID starting with 15a7fbe6006e93c464bbb14139fc42879f572f9555b2b3d8904ab7f1e35c0c2c not found: ID does not exist" containerID="15a7fbe6006e93c464bbb14139fc42879f572f9555b2b3d8904ab7f1e35c0c2c" Jul 09 05:10:15.320834 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:15.320804 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15a7fbe6006e93c464bbb14139fc42879f572f9555b2b3d8904ab7f1e35c0c2c"} err="failed to get container status \"15a7fbe6006e93c464bbb14139fc42879f572f9555b2b3d8904ab7f1e35c0c2c\": rpc error: code = NotFound desc = could not find container \"15a7fbe6006e93c464bbb14139fc42879f572f9555b2b3d8904ab7f1e35c0c2c\": container with ID starting with 15a7fbe6006e93c464bbb14139fc42879f572f9555b2b3d8904ab7f1e35c0c2c not found: ID does not exist" Jul 09 05:10:15.324217 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:15.324194 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52"] Jul 09 05:10:15.328261 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:15.328239 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-33798d-openshift-default-5d7f7d849f-wxz52"] Jul 09 05:10:15.366862 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:15.366832 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-94ccb68c4-j2rhb_8857ee0d-be11-43d5-9732-8418776e2f22/manager/0.log" Jul 09 05:10:15.376832 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:15.376778 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-94ccb68c4-j2rhb_8857ee0d-be11-43d5-9732-8418776e2f22/copy-manifests/0.log" Jul 09 05:10:16.196458 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:16.196419 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a" path="/var/lib/kubelet/pods/4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a/volumes" Jul 09 05:10:16.627595 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:16.627520 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5b5d8996bc-hwqqk_e79be3a5-b853-4c59-95c6-7a914490b429/authorino/0.log" Jul 09 05:10:16.743668 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:16.743627 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-jmqxw_1d55e992-8593-4288-b65c-29adcb45c42a/manager/0.log" Jul 09 05:10:16.851575 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:16.851546 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-p6wwc_3def7d50-d793-4f7b-bdd9-6c815bfe6b98/manager/1.log" Jul 09 05:10:16.959495 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:16.959462 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-fw8fm_c57010ee-79c3-4870-b65b-c5408ab24ef7/kuadrant-console-plugin/0.log" Jul 09 05:10:17.072781 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:17.072750 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-2mqz2_3f08431a-d380-4d64-90c5-cea110be4794/registry-server/0.log" Jul 09 05:10:17.415553 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:17.415524 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-zzrz7_514ee074-905b-413c-b139-aec3fabc856c/manager/0.log" Jul 09 05:10:17.975655 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:17.975625 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c47bc47d8-7hn5h_90ef9e78-acbb-4003-977b-0e8dfa947e8f/kube-auth-proxy/0.log" Jul 09 05:10:18.203319 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:18.203289 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-kwvjl_ae4bac90-3e35-4e14-ac3d-126c2f7d274b/istio-proxy/1.log" Jul 09 05:10:18.315697 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:18.315613 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-w9nq6_762233b1-a5ab-46d1-ad87-e1a2c2c4bd05/payload-pre-processing/0.log" Jul 09 05:10:18.426327 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:18.426296 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-wjbsj_e8d4c4c2-a870-4168-ac0e-6281adc1115d/payload-processing/0.log" Jul 09 05:10:18.873891 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:18.873858 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-gbp5t_46e18624-cb9b-436b-89f0-2dac0768866e/main/0.log" Jul 09 05:10:18.982932 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:18.982905 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-nc8tp_1bc566c7-fdb5-427c-8718-2b5cd9a58998/main/0.log" Jul 09 05:10:19.316110 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:19.316085 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-4hnxm_db73bba3-8398-4c4f-b2fc-b90d8d28e66a/main/0.log" Jul 09 05:10:19.419776 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:19.419748 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-xdrzk_69c53279-0331-4154-a29d-68b283c0e741/main/0.log" Jul 09 05:10:23.152092 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:23.152060 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:10:23.253327 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:23.253290 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" limit=1024 Jul 09 05:10:26.023135 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:26.023107 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-zl4tp_d0a12812-bbc1-4b88-a2f2-858d22d67e64/global-pull-secret-syncer/0.log" Jul 09 05:10:26.069202 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:26.069167 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-fltwp_97d22a8e-85f8-4d3a-8fbf-cdff7581f85e/konnectivity-agent/0.log" Jul 09 05:10:26.180853 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:26.180823 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-184.ec2.internal_be23e120c66897ba7a1764c4c10c1b91/haproxy/0.log" Jul 09 05:10:31.726971 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:31.726935 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5b5d8996bc-hwqqk_e79be3a5-b853-4c59-95c6-7a914490b429/authorino/0.log" Jul 09 05:10:31.751303 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:31.751269 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-jmqxw_1d55e992-8593-4288-b65c-29adcb45c42a/manager/0.log" Jul 09 05:10:31.776009 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:31.775983 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-p6wwc_3def7d50-d793-4f7b-bdd9-6c815bfe6b98/manager/1.log" Jul 09 05:10:31.776220 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:31.776081 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-p6wwc_3def7d50-d793-4f7b-bdd9-6c815bfe6b98/manager/0.log" Jul 09 05:10:31.795624 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:31.795601 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-fw8fm_c57010ee-79c3-4870-b65b-c5408ab24ef7/kuadrant-console-plugin/0.log" Jul 09 05:10:31.831671 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:31.831640 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-2mqz2_3f08431a-d380-4d64-90c5-cea110be4794/registry-server/0.log" Jul 09 05:10:32.140619 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:32.140525 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-zzrz7_514ee074-905b-413c-b139-aec3fabc856c/manager/0.log" Jul 09 05:10:34.144148 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:34.144108 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-mvq75_ffccf1f2-c6d1-4d17-8387-a38651058f81/kube-state-metrics/0.log" Jul 09 05:10:34.163938 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:34.163892 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-mvq75_ffccf1f2-c6d1-4d17-8387-a38651058f81/kube-rbac-proxy-main/0.log" Jul 09 05:10:34.184451 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:34.184413 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-mvq75_ffccf1f2-c6d1-4d17-8387-a38651058f81/kube-rbac-proxy-self/0.log" Jul 09 05:10:34.339147 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:34.339117 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6slcp_3867ddde-c7e8-4e28-b44f-ce053ff6e7f9/node-exporter/0.log" Jul 09 05:10:34.364405 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:34.364367 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6slcp_3867ddde-c7e8-4e28-b44f-ce053ff6e7f9/kube-rbac-proxy/0.log" Jul 09 05:10:34.389558 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:34.389533 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6slcp_3867ddde-c7e8-4e28-b44f-ce053ff6e7f9/init-textfile/0.log" Jul 09 05:10:34.494588 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:34.494551 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-swnzm_10318b5f-09fb-4670-9c39-a6f028c961fb/kube-rbac-proxy-main/0.log" Jul 09 05:10:34.517490 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:34.517451 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-swnzm_10318b5f-09fb-4670-9c39-a6f028c961fb/kube-rbac-proxy-self/0.log" Jul 09 05:10:34.539664 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:34.539633 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-swnzm_10318b5f-09fb-4670-9c39-a6f028c961fb/openshift-state-metrics/0.log" Jul 09 05:10:36.232898 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:36.232866 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ts86v/perf-node-gather-daemonset-948wk"] Jul 09 05:10:36.233284 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:36.233269 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a" containerName="istio-proxy" Jul 09 05:10:36.233360 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:36.233286 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a" containerName="istio-proxy" Jul 09 05:10:36.233440 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:36.233426 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a" containerName="istio-proxy" Jul 09 05:10:36.233476 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:36.233447 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b4b6fad-bb26-4132-a0aa-cfdab59e1b1a" containerName="istio-proxy" Jul 09 05:10:36.238076 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:36.238054 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ts86v/perf-node-gather-daemonset-948wk" Jul 09 05:10:36.240512 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:36.240488 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ts86v\"/\"openshift-service-ca.crt\"" Jul 09 05:10:36.241518 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:36.241487 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ts86v\"/\"kube-root-ca.crt\"" Jul 09 05:10:36.241642 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:36.241528 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ts86v\"/\"default-dockercfg-lmq77\"" Jul 09 05:10:36.243566 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:36.243546 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ts86v/perf-node-gather-daemonset-948wk"] Jul 09 05:10:36.359161 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:36.359125 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbjj8\" (UniqueName: \"kubernetes.io/projected/d875bf04-07dd-477a-8273-826e96bc64e5-kube-api-access-mbjj8\") pod \"perf-node-gather-daemonset-948wk\" (UID: \"d875bf04-07dd-477a-8273-826e96bc64e5\") " pod="openshift-must-gather-ts86v/perf-node-gather-daemonset-948wk" Jul 09 05:10:36.359389 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:36.359176 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d875bf04-07dd-477a-8273-826e96bc64e5-lib-modules\") pod \"perf-node-gather-daemonset-948wk\" (UID: \"d875bf04-07dd-477a-8273-826e96bc64e5\") " pod="openshift-must-gather-ts86v/perf-node-gather-daemonset-948wk" Jul 09 05:10:36.359389 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:36.359301 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d875bf04-07dd-477a-8273-826e96bc64e5-sys\") pod \"perf-node-gather-daemonset-948wk\" (UID: \"d875bf04-07dd-477a-8273-826e96bc64e5\") " pod="openshift-must-gather-ts86v/perf-node-gather-daemonset-948wk" Jul 09 05:10:36.359526 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:36.359401 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d875bf04-07dd-477a-8273-826e96bc64e5-proc\") pod \"perf-node-gather-daemonset-948wk\" (UID: \"d875bf04-07dd-477a-8273-826e96bc64e5\") " pod="openshift-must-gather-ts86v/perf-node-gather-daemonset-948wk" Jul 09 05:10:36.359526 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:36.359426 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d875bf04-07dd-477a-8273-826e96bc64e5-podres\") pod \"perf-node-gather-daemonset-948wk\" (UID: \"d875bf04-07dd-477a-8273-826e96bc64e5\") " pod="openshift-must-gather-ts86v/perf-node-gather-daemonset-948wk" Jul 09 05:10:36.460810 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:36.460780 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d875bf04-07dd-477a-8273-826e96bc64e5-proc\") pod \"perf-node-gather-daemonset-948wk\" (UID: \"d875bf04-07dd-477a-8273-826e96bc64e5\") " pod="openshift-must-gather-ts86v/perf-node-gather-daemonset-948wk" Jul 09 05:10:36.460810 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:36.460813 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d875bf04-07dd-477a-8273-826e96bc64e5-podres\") pod \"perf-node-gather-daemonset-948wk\" (UID: \"d875bf04-07dd-477a-8273-826e96bc64e5\") " pod="openshift-must-gather-ts86v/perf-node-gather-daemonset-948wk" Jul 09 05:10:36.461057 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:36.460872 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbjj8\" (UniqueName: \"kubernetes.io/projected/d875bf04-07dd-477a-8273-826e96bc64e5-kube-api-access-mbjj8\") pod \"perf-node-gather-daemonset-948wk\" (UID: \"d875bf04-07dd-477a-8273-826e96bc64e5\") " pod="openshift-must-gather-ts86v/perf-node-gather-daemonset-948wk" Jul 09 05:10:36.461057 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:36.460926 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d875bf04-07dd-477a-8273-826e96bc64e5-lib-modules\") pod \"perf-node-gather-daemonset-948wk\" (UID: \"d875bf04-07dd-477a-8273-826e96bc64e5\") " pod="openshift-must-gather-ts86v/perf-node-gather-daemonset-948wk" Jul 09 05:10:36.461057 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:36.460975 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d875bf04-07dd-477a-8273-826e96bc64e5-podres\") pod \"perf-node-gather-daemonset-948wk\" (UID: \"d875bf04-07dd-477a-8273-826e96bc64e5\") " pod="openshift-must-gather-ts86v/perf-node-gather-daemonset-948wk" Jul 09 05:10:36.461057 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:36.460995 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d875bf04-07dd-477a-8273-826e96bc64e5-sys\") pod \"perf-node-gather-daemonset-948wk\" (UID: \"d875bf04-07dd-477a-8273-826e96bc64e5\") " pod="openshift-must-gather-ts86v/perf-node-gather-daemonset-948wk" Jul 09 05:10:36.461057 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:36.460927 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d875bf04-07dd-477a-8273-826e96bc64e5-proc\") pod \"perf-node-gather-daemonset-948wk\" (UID: \"d875bf04-07dd-477a-8273-826e96bc64e5\") " pod="openshift-must-gather-ts86v/perf-node-gather-daemonset-948wk" Jul 09 05:10:36.461315 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:36.461059 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d875bf04-07dd-477a-8273-826e96bc64e5-sys\") pod \"perf-node-gather-daemonset-948wk\" (UID: \"d875bf04-07dd-477a-8273-826e96bc64e5\") " pod="openshift-must-gather-ts86v/perf-node-gather-daemonset-948wk" Jul 09 05:10:36.461315 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:36.461061 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d875bf04-07dd-477a-8273-826e96bc64e5-lib-modules\") pod \"perf-node-gather-daemonset-948wk\" (UID: \"d875bf04-07dd-477a-8273-826e96bc64e5\") " pod="openshift-must-gather-ts86v/perf-node-gather-daemonset-948wk" Jul 09 05:10:36.468505 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:36.468466 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbjj8\" (UniqueName: \"kubernetes.io/projected/d875bf04-07dd-477a-8273-826e96bc64e5-kube-api-access-mbjj8\") pod \"perf-node-gather-daemonset-948wk\" (UID: \"d875bf04-07dd-477a-8273-826e96bc64e5\") " pod="openshift-must-gather-ts86v/perf-node-gather-daemonset-948wk" Jul 09 05:10:36.549589 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:36.549492 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ts86v/perf-node-gather-daemonset-948wk" Jul 09 05:10:36.679651 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:36.679623 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ts86v/perf-node-gather-daemonset-948wk"] Jul 09 05:10:36.681898 ip-10-0-139-184 kubenswrapper[2565]: W0709 05:10:36.681875 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd875bf04_07dd_477a_8273_826e96bc64e5.slice/crio-38a377674e3605bef81d68df5941d7e44c16d35ee801993aee9814bcdab4ca74 WatchSource:0}: Error finding container 38a377674e3605bef81d68df5941d7e44c16d35ee801993aee9814bcdab4ca74: Status 404 returned error can't find the container with id 38a377674e3605bef81d68df5941d7e44c16d35ee801993aee9814bcdab4ca74 Jul 09 05:10:37.412526 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:37.412490 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ts86v/perf-node-gather-daemonset-948wk" event={"ID":"d875bf04-07dd-477a-8273-826e96bc64e5","Type":"ContainerStarted","Data":"5aa4b3329fb19a1ab0a2a3c7a9f2cacf5f1556e22429d187e314b8a2b0f50381"} Jul 09 05:10:37.412947 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:37.412539 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ts86v/perf-node-gather-daemonset-948wk" event={"ID":"d875bf04-07dd-477a-8273-826e96bc64e5","Type":"ContainerStarted","Data":"38a377674e3605bef81d68df5941d7e44c16d35ee801993aee9814bcdab4ca74"} Jul 09 05:10:37.412947 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:37.412613 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-ts86v/perf-node-gather-daemonset-948wk" Jul 09 05:10:37.428245 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:37.428199 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ts86v/perf-node-gather-daemonset-948wk" podStartSLOduration=1.428185009 podStartE2EDuration="1.428185009s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 05:10:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 05:10:37.425888119 +0000 UTC m=+2309.677411697" watchObservedRunningTime="2026-07-09 05:10:37.428185009 +0000 UTC m=+2309.679708587" Jul 09 05:10:37.484402 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:37.484378 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-554bb9dccb-bz2vd_d99c210e-193f-416e-9c5e-234f96d978cc/console/0.log" Jul 09 05:10:39.139962 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:39.139933 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ct9t6_7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766/dns/0.log" Jul 09 05:10:39.160360 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:39.160314 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ct9t6_7a6fbfe4-8f20-4ae6-98bd-9a197b6b5766/kube-rbac-proxy/0.log" Jul 09 05:10:39.280496 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:39.280467 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xzgbm_7b2795d7-ba4b-46ea-bd2c-6b9d061e9746/dns-node-resolver/0.log" Jul 09 05:10:39.825616 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:39.825584 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-76b76c5bc7-4r6xd_974fb6f3-fe6f-4ea7-9987-1813f7f037a1/registry/0.log" Jul 09 05:10:39.849053 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:39.849024 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-j7zhv_227fe7b3-97ee-435b-9e07-15cf67dd2beb/node-ca/0.log" Jul 09 05:10:40.937620 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:40.937587 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c47bc47d8-7hn5h_90ef9e78-acbb-4003-977b-0e8dfa947e8f/kube-auth-proxy/0.log" Jul 09 05:10:40.990233 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:40.990205 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-kwvjl_ae4bac90-3e35-4e14-ac3d-126c2f7d274b/istio-proxy/0.log" Jul 09 05:10:41.016819 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:41.016789 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-kwvjl_ae4bac90-3e35-4e14-ac3d-126c2f7d274b/istio-proxy/1.log" Jul 09 05:10:41.052116 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:41.052085 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-w9nq6_762233b1-a5ab-46d1-ad87-e1a2c2c4bd05/payload-pre-processing/0.log" Jul 09 05:10:41.096698 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:41.096669 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-wjbsj_e8d4c4c2-a870-4168-ac0e-6281adc1115d/payload-processing/0.log" Jul 09 05:10:41.830687 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:41.830659 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wbngq_964ab4da-c25d-46bd-97df-ffb2cd5b820b/serve-healthcheck-canary/0.log" Jul 09 05:10:42.457459 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:42.457436 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xgznh_359f3d6e-a08a-487d-a480-a642dad64ec4/kube-rbac-proxy/0.log" Jul 09 05:10:42.476951 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:42.476925 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xgznh_359f3d6e-a08a-487d-a480-a642dad64ec4/exporter/0.log" Jul 09 05:10:42.498147 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:42.498121 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xgznh_359f3d6e-a08a-487d-a480-a642dad64ec4/extractor/0.log" Jul 09 05:10:43.427431 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:43.427402 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-ts86v/perf-node-gather-daemonset-948wk" Jul 09 05:10:44.585781 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:44.585747 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-mx8xg_02072840-9f3b-43a1-a940-ac8aa8b1c188/manager/0.log" Jul 09 05:10:44.691202 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:44.691172 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-x8wrk_bed406fb-4a82-40d6-b190-5cf5033f6867/manager/0.log" Jul 09 05:10:44.781321 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:44.781296 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-d78b58cc-xvgdd_dd160605-cb97-41bb-8283-f5545fd734f6/manager/0.log" Jul 09 05:10:44.801287 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:44.801258 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-9rgfm_91e3e24b-5366-4fea-a415-cd9e2bb46439/server/0.log" Jul 09 05:10:44.858953 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:44.858872 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-lzxrb_fb4b96ac-6cd9-4b45-a2b9-89667201bb54/manager/0.log" Jul 09 05:10:45.072418 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:45.072375 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-94ccb68c4-j2rhb_8857ee0d-be11-43d5-9732-8418776e2f22/manager/0.log" Jul 09 05:10:45.088572 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:45.088550 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-94ccb68c4-j2rhb_8857ee0d-be11-43d5-9732-8418776e2f22/copy-manifests/0.log" Jul 09 05:10:47.010045 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:47.010016 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c47bc47d8-7hn5h_90ef9e78-acbb-4003-977b-0e8dfa947e8f/kube-auth-proxy/0.log" Jul 09 05:10:47.046029 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:47.046004 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-kwvjl_ae4bac90-3e35-4e14-ac3d-126c2f7d274b/istio-proxy/0.log" Jul 09 05:10:47.072650 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:47.072623 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-kwvjl_ae4bac90-3e35-4e14-ac3d-126c2f7d274b/istio-proxy/1.log" Jul 09 05:10:47.101235 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:47.101205 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-w9nq6_762233b1-a5ab-46d1-ad87-e1a2c2c4bd05/payload-pre-processing/0.log" Jul 09 05:10:47.135808 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:47.135764 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-wjbsj_e8d4c4c2-a870-4168-ac0e-6281adc1115d/payload-processing/0.log" Jul 09 05:10:48.509853 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:48.509826 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c47bc47d8-7hn5h_90ef9e78-acbb-4003-977b-0e8dfa947e8f/kube-auth-proxy/0.log" Jul 09 05:10:48.548834 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:48.548805 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-kwvjl_ae4bac90-3e35-4e14-ac3d-126c2f7d274b/istio-proxy/0.log" Jul 09 05:10:48.575439 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:48.575404 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-kwvjl_ae4bac90-3e35-4e14-ac3d-126c2f7d274b/istio-proxy/1.log" Jul 09 05:10:48.608523 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:48.608495 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-w9nq6_762233b1-a5ab-46d1-ad87-e1a2c2c4bd05/payload-pre-processing/0.log" Jul 09 05:10:48.644163 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:48.644111 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-wjbsj_e8d4c4c2-a870-4168-ac0e-6281adc1115d/payload-processing/0.log" Jul 09 05:10:52.759001 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:52.758971 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2tgfm_2b4a7d33-f06c-489c-ab25-b7b5b109d33c/kube-multus/0.log" Jul 09 05:10:52.785614 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:52.785590 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cmpgg_af45d649-376b-4efd-ad4c-9b5b3c28ab4d/kube-multus-additional-cni-plugins/0.log" Jul 09 05:10:52.806072 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:52.806045 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cmpgg_af45d649-376b-4efd-ad4c-9b5b3c28ab4d/egress-router-binary-copy/0.log" Jul 09 05:10:52.828245 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:52.828221 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cmpgg_af45d649-376b-4efd-ad4c-9b5b3c28ab4d/cni-plugins/0.log" Jul 09 05:10:52.848436 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:52.848414 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cmpgg_af45d649-376b-4efd-ad4c-9b5b3c28ab4d/bond-cni-plugin/0.log" Jul 09 05:10:52.869725 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:52.869695 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cmpgg_af45d649-376b-4efd-ad4c-9b5b3c28ab4d/routeoverride-cni/0.log" Jul 09 05:10:52.889650 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:52.889623 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cmpgg_af45d649-376b-4efd-ad4c-9b5b3c28ab4d/whereabouts-cni-bincopy/0.log" Jul 09 05:10:52.909822 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:52.909796 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cmpgg_af45d649-376b-4efd-ad4c-9b5b3c28ab4d/whereabouts-cni/0.log" Jul 09 05:10:53.314926 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:53.314897 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-gpx47_dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa/network-metrics-daemon/0.log" Jul 09 05:10:53.336666 ip-10-0-139-184 kubenswrapper[2565]: I0709 05:10:53.336634 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-gpx47_dc88f9f0-1d2f-4522-b7ca-d0b9104f85fa/kube-rbac-proxy/0.log"