Jul 02 21:36:01.374802 ip-10-0-140-146 systemd[1]: Starting Kubernetes Kubelet... Jul 02 21:36:01.805224 ip-10-0-140-146 kubenswrapper[2556]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 21:36:01.805224 ip-10-0-140-146 kubenswrapper[2556]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 02 21:36:01.805224 ip-10-0-140-146 kubenswrapper[2556]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 21:36:01.805224 ip-10-0-140-146 kubenswrapper[2556]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 02 21:36:01.805224 ip-10-0-140-146 kubenswrapper[2556]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 21:36:01.805909 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.805286 2556 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 02 21:36:01.809980 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.809965 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 21:36:01.809980 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.809979 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 21:36:01.810041 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.809983 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 21:36:01.810041 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.809986 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 21:36:01.810041 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.809989 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 21:36:01.810041 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.809992 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 21:36:01.810041 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.809996 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 21:36:01.810041 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.809999 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 21:36:01.810041 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810001 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 21:36:01.810041 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810004 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 21:36:01.810041 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810007 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 21:36:01.810041 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810009 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 21:36:01.810041 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810012 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 21:36:01.810041 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810021 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 21:36:01.810041 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810024 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 21:36:01.810041 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810027 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 21:36:01.810041 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810029 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 21:36:01.810041 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810032 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 21:36:01.810041 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810034 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 21:36:01.810041 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810037 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 21:36:01.810041 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810039 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 21:36:01.810041 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810042 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 21:36:01.810564 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810045 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 21:36:01.810564 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810048 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 21:36:01.810564 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810051 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 21:36:01.810564 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810054 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 21:36:01.810564 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810056 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 21:36:01.810564 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810059 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 21:36:01.810564 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810062 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 21:36:01.810564 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810064 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 21:36:01.810564 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810067 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 21:36:01.810564 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810069 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 21:36:01.810564 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810072 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 21:36:01.810564 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810075 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 21:36:01.810564 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810077 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 21:36:01.810564 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810079 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 21:36:01.810564 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810082 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 21:36:01.810564 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810084 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 21:36:01.810564 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810086 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 21:36:01.810564 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810089 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 21:36:01.810564 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810092 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 21:36:01.810564 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810094 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 21:36:01.810564 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810096 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 21:36:01.811076 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810099 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 21:36:01.811076 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810101 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 21:36:01.811076 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810104 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 21:36:01.811076 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810107 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 21:36:01.811076 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810109 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 21:36:01.811076 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810112 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 21:36:01.811076 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810114 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 21:36:01.811076 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810117 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 21:36:01.811076 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810119 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 21:36:01.811076 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810122 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 21:36:01.811076 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810124 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 21:36:01.811076 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810127 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 21:36:01.811076 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810130 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 21:36:01.811076 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810133 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 21:36:01.811076 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810135 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 21:36:01.811076 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810138 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 21:36:01.811076 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810140 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 21:36:01.811076 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810143 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 21:36:01.811076 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810145 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 21:36:01.811584 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810148 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 21:36:01.811584 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810150 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 21:36:01.811584 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810153 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 21:36:01.811584 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810155 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 21:36:01.811584 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810158 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 21:36:01.811584 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810160 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 21:36:01.811584 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810162 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 21:36:01.811584 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810167 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 21:36:01.811584 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810171 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 21:36:01.811584 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810174 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 21:36:01.811584 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810176 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 21:36:01.811584 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810179 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 21:36:01.811584 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810181 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 21:36:01.811584 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810184 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 21:36:01.811584 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810187 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 21:36:01.811584 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810191 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 21:36:01.811584 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810194 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 21:36:01.811584 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810197 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 21:36:01.812035 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810200 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 21:36:01.812035 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810203 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 21:36:01.812035 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810206 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 21:36:01.812035 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810208 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 21:36:01.812035 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810211 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 21:36:01.812035 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810214 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 21:36:01.812035 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810216 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 21:36:01.812035 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810220 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 21:36:01.812035 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810223 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 21:36:01.812035 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810226 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 21:36:01.812035 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810230 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 21:36:01.812035 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810233 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 21:36:01.812035 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810236 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 21:36:01.812035 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810239 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 21:36:01.812035 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810242 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 21:36:01.812035 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810245 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 21:36:01.812035 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810248 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 21:36:01.812035 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810251 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 02 21:36:01.812035 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810254 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 21:36:01.812560 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810987 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 21:36:01.812560 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810993 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 21:36:01.812560 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810996 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 21:36:01.812560 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.810999 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 02 21:36:01.812560 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811002 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 21:36:01.812560 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811005 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 21:36:01.812560 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811007 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 21:36:01.812560 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811010 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 21:36:01.812560 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811013 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 21:36:01.812560 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811015 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 21:36:01.812560 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811018 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 21:36:01.812560 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811020 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 21:36:01.812560 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811024 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 21:36:01.812560 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811026 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 21:36:01.812560 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811030 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 21:36:01.812560 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811032 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 21:36:01.812560 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811036 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 21:36:01.812560 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811039 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 21:36:01.812560 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811043 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 21:36:01.812560 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811046 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 21:36:01.813053 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811049 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 21:36:01.813053 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811051 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 21:36:01.813053 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811054 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 21:36:01.813053 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811056 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 21:36:01.813053 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811059 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 21:36:01.813053 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811062 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 21:36:01.813053 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811064 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 21:36:01.813053 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811067 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 21:36:01.813053 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811069 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 21:36:01.813053 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811072 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 21:36:01.813053 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811075 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 21:36:01.813053 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811078 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 21:36:01.813053 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811080 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 21:36:01.813053 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811083 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 21:36:01.813053 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811086 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 21:36:01.813053 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811088 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 21:36:01.813053 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811091 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 21:36:01.813053 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811093 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 21:36:01.813053 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811096 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 21:36:01.813053 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811098 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 21:36:01.813691 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811101 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 21:36:01.813691 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811103 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 21:36:01.813691 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811106 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 21:36:01.813691 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811108 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 21:36:01.813691 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811111 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 21:36:01.813691 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811115 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 21:36:01.813691 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811119 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 21:36:01.813691 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811122 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 21:36:01.813691 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811125 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 21:36:01.813691 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811128 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 21:36:01.813691 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811131 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 21:36:01.813691 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811134 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 21:36:01.813691 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811137 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 21:36:01.813691 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811140 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 21:36:01.813691 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811142 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 21:36:01.813691 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811145 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 21:36:01.813691 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811147 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 21:36:01.813691 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811150 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 21:36:01.813691 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811152 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 21:36:01.813691 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811155 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 21:36:01.814186 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811157 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 21:36:01.814186 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811160 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 21:36:01.814186 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811162 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 21:36:01.814186 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811165 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 21:36:01.814186 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811167 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 21:36:01.814186 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811170 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 21:36:01.814186 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811172 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 21:36:01.814186 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811175 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 21:36:01.814186 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811177 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 21:36:01.814186 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811180 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 21:36:01.814186 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811184 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 21:36:01.814186 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811186 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 21:36:01.814186 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811189 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 21:36:01.814186 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811191 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 21:36:01.814186 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811194 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 21:36:01.814186 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811196 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 21:36:01.814186 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811199 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 21:36:01.814186 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811201 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 21:36:01.814186 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811205 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 21:36:01.814186 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811207 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 21:36:01.814694 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811211 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 21:36:01.814694 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811213 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 21:36:01.814694 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811215 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 21:36:01.814694 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811218 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 21:36:01.814694 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811220 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 21:36:01.814694 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811223 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 21:36:01.814694 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811226 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 21:36:01.814694 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811230 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 21:36:01.814694 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811232 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 21:36:01.814694 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811235 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 21:36:01.814694 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811238 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 21:36:01.814694 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811241 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 21:36:01.814694 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811243 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 21:36:01.814694 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811245 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 21:36:01.814694 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811248 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 21:36:01.814694 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811250 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 21:36:01.814694 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811252 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 21:36:01.814694 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811255 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 21:36:01.814694 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811257 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 21:36:01.815149 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811675 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 21:36:01.815149 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811682 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 21:36:01.815149 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811685 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 21:36:01.815149 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811689 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 21:36:01.815149 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811692 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 21:36:01.815149 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811695 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 21:36:01.815149 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811697 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 21:36:01.815149 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811700 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 21:36:01.815149 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811703 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 21:36:01.815149 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811706 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 21:36:01.815149 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811709 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 21:36:01.815149 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811712 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 21:36:01.815149 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811714 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 21:36:01.815149 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811717 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 21:36:01.815149 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811719 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 21:36:01.815149 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811722 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 21:36:01.815149 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811725 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 21:36:01.815149 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811727 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 21:36:01.815149 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811730 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 21:36:01.815149 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811732 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 21:36:01.815638 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811735 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 21:36:01.815638 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811737 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 21:36:01.815638 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811740 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 21:36:01.815638 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811742 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 21:36:01.815638 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811745 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 21:36:01.815638 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811748 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 02 21:36:01.815638 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811750 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 21:36:01.815638 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811753 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 21:36:01.815638 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811755 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 21:36:01.815638 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811758 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 21:36:01.815638 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811761 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 21:36:01.815638 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811763 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 21:36:01.815638 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811765 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 21:36:01.815638 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811768 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 21:36:01.815638 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811771 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 21:36:01.815638 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811773 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 21:36:01.815638 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811776 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 21:36:01.815638 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811779 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 21:36:01.815638 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811783 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 21:36:01.815638 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811787 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 21:36:01.816128 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811792 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 21:36:01.816128 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811795 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 21:36:01.816128 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811798 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 21:36:01.816128 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811800 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 21:36:01.816128 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811803 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 21:36:01.816128 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811806 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 21:36:01.816128 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811809 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 21:36:01.816128 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811811 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 21:36:01.816128 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811814 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 21:36:01.816128 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811816 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 21:36:01.816128 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811819 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 21:36:01.816128 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811821 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 21:36:01.816128 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811824 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 21:36:01.816128 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811826 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 21:36:01.816128 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811829 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 21:36:01.816128 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811831 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 21:36:01.816128 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811833 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 21:36:01.816128 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811836 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 21:36:01.816128 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811838 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 21:36:01.816606 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811841 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 21:36:01.816606 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811844 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 21:36:01.816606 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811847 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 21:36:01.816606 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811850 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 21:36:01.816606 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811853 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 21:36:01.816606 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811856 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 21:36:01.816606 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811859 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 21:36:01.816606 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811862 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 21:36:01.816606 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811865 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 21:36:01.816606 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811867 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 21:36:01.816606 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811870 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 21:36:01.816606 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811873 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 21:36:01.816606 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811875 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 21:36:01.816606 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811879 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 21:36:01.816606 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811881 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 21:36:01.816606 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811884 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 21:36:01.816606 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811886 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 21:36:01.816606 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811889 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 21:36:01.816606 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811891 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 21:36:01.817068 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811894 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 21:36:01.817068 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811896 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 21:36:01.817068 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811899 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 21:36:01.817068 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811901 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 21:36:01.817068 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811905 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 21:36:01.817068 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811907 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 21:36:01.817068 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811909 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 21:36:01.817068 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811912 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 21:36:01.817068 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811915 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 21:36:01.817068 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811917 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 21:36:01.817068 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811920 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 21:36:01.817068 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811922 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 21:36:01.817068 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811925 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 21:36:01.817068 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811927 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 21:36:01.817068 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811930 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 21:36:01.817068 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811932 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 21:36:01.817068 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811935 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 21:36:01.817068 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811937 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 21:36:01.817068 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811939 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 21:36:01.817068 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811944 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 21:36:01.817571 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.811946 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 21:36:01.817571 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812005 2556 flags.go:64] FLAG: --address="0.0.0.0" Jul 02 21:36:01.817571 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812011 2556 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 02 21:36:01.817571 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812017 2556 flags.go:64] FLAG: --anonymous-auth="true" Jul 02 21:36:01.817571 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812021 2556 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 02 21:36:01.817571 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812025 2556 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 02 21:36:01.817571 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812028 2556 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 02 21:36:01.817571 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812032 2556 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 02 21:36:01.817571 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812037 2556 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 02 21:36:01.817571 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812040 2556 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 02 21:36:01.817571 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812043 2556 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 02 21:36:01.817571 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812046 2556 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 02 21:36:01.817571 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812050 2556 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 02 21:36:01.817571 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812054 2556 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 02 21:36:01.817571 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812056 2556 flags.go:64] FLAG: --cgroup-root="" Jul 02 21:36:01.817571 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812059 2556 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 02 21:36:01.817571 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812062 2556 flags.go:64] FLAG: --client-ca-file="" Jul 02 21:36:01.817571 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812065 2556 flags.go:64] FLAG: --cloud-provider="external" Jul 02 21:36:01.817571 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812068 2556 flags.go:64] FLAG: --cluster-dns="[]" Jul 02 21:36:01.817571 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812073 2556 flags.go:64] FLAG: --cluster-domain="" Jul 02 21:36:01.817571 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812076 2556 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 02 21:36:01.817571 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812079 2556 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 02 21:36:01.817571 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812082 2556 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 02 21:36:01.817571 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812085 2556 flags.go:64] FLAG: --container-log-max-files="5" Jul 02 21:36:01.818171 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812089 2556 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 02 21:36:01.818171 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812092 2556 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 02 21:36:01.818171 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812095 2556 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 02 21:36:01.818171 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812098 2556 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 02 21:36:01.818171 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812101 2556 flags.go:64] FLAG: --contention-profiling="false" Jul 02 21:36:01.818171 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812104 2556 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 02 21:36:01.818171 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812107 2556 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 02 21:36:01.818171 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812111 2556 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 02 21:36:01.818171 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812114 2556 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 02 21:36:01.818171 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812118 2556 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 02 21:36:01.818171 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812121 2556 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 02 21:36:01.818171 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812124 2556 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 02 21:36:01.818171 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812126 2556 flags.go:64] FLAG: --enable-load-reader="false" Jul 02 21:36:01.818171 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812129 2556 flags.go:64] FLAG: --enable-server="true" Jul 02 21:36:01.818171 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812132 2556 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 02 21:36:01.818171 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812136 2556 flags.go:64] FLAG: --event-burst="100" Jul 02 21:36:01.818171 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812139 2556 flags.go:64] FLAG: --event-qps="50" Jul 02 21:36:01.818171 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812142 2556 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 02 21:36:01.818171 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812144 2556 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 02 21:36:01.818171 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812147 2556 flags.go:64] FLAG: --eviction-hard="" Jul 02 21:36:01.818171 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812151 2556 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 02 21:36:01.818171 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812158 2556 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 02 21:36:01.818171 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812162 2556 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 02 21:36:01.818171 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812165 2556 flags.go:64] FLAG: --eviction-soft="" Jul 02 21:36:01.818171 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812168 2556 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 02 21:36:01.818803 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812171 2556 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 02 21:36:01.818803 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812174 2556 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 02 21:36:01.818803 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812177 2556 flags.go:64] FLAG: --experimental-mounter-path="" Jul 02 21:36:01.818803 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812180 2556 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 02 21:36:01.818803 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812183 2556 flags.go:64] FLAG: --fail-swap-on="true" Jul 02 21:36:01.818803 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812185 2556 flags.go:64] FLAG: --feature-gates="" Jul 02 21:36:01.818803 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812189 2556 flags.go:64] FLAG: --file-check-frequency="20s" Jul 02 21:36:01.818803 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812192 2556 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 02 21:36:01.818803 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812195 2556 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 02 21:36:01.818803 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812198 2556 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 02 21:36:01.818803 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812201 2556 flags.go:64] FLAG: --healthz-port="10248" Jul 02 21:36:01.818803 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812203 2556 flags.go:64] FLAG: --help="false" Jul 02 21:36:01.818803 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812206 2556 flags.go:64] FLAG: --hostname-override="ip-10-0-140-146.ec2.internal" Jul 02 21:36:01.818803 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812209 2556 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 02 21:36:01.818803 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812213 2556 flags.go:64] FLAG: --http-check-frequency="20s" Jul 02 21:36:01.818803 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812216 2556 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 02 21:36:01.818803 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812219 2556 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 02 21:36:01.818803 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812222 2556 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 02 21:36:01.818803 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812225 2556 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 02 21:36:01.818803 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812227 2556 flags.go:64] FLAG: --image-service-endpoint="" Jul 02 21:36:01.818803 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812230 2556 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 02 21:36:01.818803 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812233 2556 flags.go:64] FLAG: --kube-api-burst="100" Jul 02 21:36:01.818803 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812236 2556 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 02 21:36:01.819531 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812239 2556 flags.go:64] FLAG: --kube-api-qps="50" Jul 02 21:36:01.819531 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812242 2556 flags.go:64] FLAG: --kube-reserved="" Jul 02 21:36:01.819531 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812244 2556 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 02 21:36:01.819531 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812247 2556 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 02 21:36:01.819531 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812250 2556 flags.go:64] FLAG: --kubelet-cgroups="" Jul 02 21:36:01.819531 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812253 2556 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 02 21:36:01.819531 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812258 2556 flags.go:64] FLAG: --lock-file="" Jul 02 21:36:01.819531 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812260 2556 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 02 21:36:01.819531 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812263 2556 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 02 21:36:01.819531 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812266 2556 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 02 21:36:01.819531 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812271 2556 flags.go:64] FLAG: --log-json-split-stream="false" Jul 02 21:36:01.819531 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812274 2556 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 02 21:36:01.819531 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812277 2556 flags.go:64] FLAG: --log-text-split-stream="false" Jul 02 21:36:01.819531 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812279 2556 flags.go:64] FLAG: --logging-format="text" Jul 02 21:36:01.819531 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812282 2556 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 02 21:36:01.819531 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812285 2556 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 02 21:36:01.819531 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812288 2556 flags.go:64] FLAG: --manifest-url="" Jul 02 21:36:01.819531 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812290 2556 flags.go:64] FLAG: --manifest-url-header="" Jul 02 21:36:01.819531 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812295 2556 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 02 21:36:01.819531 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812298 2556 flags.go:64] FLAG: --max-open-files="1000000" Jul 02 21:36:01.819531 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812302 2556 flags.go:64] FLAG: --max-pods="110" Jul 02 21:36:01.819531 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812305 2556 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 02 21:36:01.819531 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812307 2556 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 02 21:36:01.819531 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812312 2556 flags.go:64] FLAG: --memory-manager-policy="None" Jul 02 21:36:01.819531 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812314 2556 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 02 21:36:01.820455 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812318 2556 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 02 21:36:01.820455 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812321 2556 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 02 21:36:01.820455 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812323 2556 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 02 21:36:01.820455 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812330 2556 flags.go:64] FLAG: --node-status-max-images="50" Jul 02 21:36:01.820455 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812333 2556 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 02 21:36:01.820455 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812336 2556 flags.go:64] FLAG: --oom-score-adj="-999" Jul 02 21:36:01.820455 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812339 2556 flags.go:64] FLAG: --pod-cidr="" Jul 02 21:36:01.820455 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812342 2556 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 02 21:36:01.820455 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812347 2556 flags.go:64] FLAG: --pod-manifest-path="" Jul 02 21:36:01.820455 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812350 2556 flags.go:64] FLAG: --pod-max-pids="-1" Jul 02 21:36:01.820455 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812353 2556 flags.go:64] FLAG: --pods-per-core="0" Jul 02 21:36:01.820455 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812356 2556 flags.go:64] FLAG: --port="10250" Jul 02 21:36:01.820455 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812358 2556 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 02 21:36:01.820455 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812363 2556 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0676a098e9bbb5d8f" Jul 02 21:36:01.820455 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812367 2556 flags.go:64] FLAG: --qos-reserved="" Jul 02 21:36:01.820455 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812370 2556 flags.go:64] FLAG: --read-only-port="10255" Jul 02 21:36:01.820455 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812372 2556 flags.go:64] FLAG: --register-node="true" Jul 02 21:36:01.820455 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812375 2556 flags.go:64] FLAG: --register-with-taints="" Jul 02 21:36:01.820455 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812378 2556 flags.go:64] FLAG: --registry-burst="10" Jul 02 21:36:01.820455 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812381 2556 flags.go:64] FLAG: --registry-qps="5" Jul 02 21:36:01.820455 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812384 2556 flags.go:64] FLAG: --reserved-cpus="" Jul 02 21:36:01.820455 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812387 2556 flags.go:64] FLAG: --reserved-memory="" Jul 02 21:36:01.820455 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812390 2556 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 02 21:36:01.820455 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812393 2556 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 02 21:36:01.820455 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812396 2556 flags.go:64] FLAG: --rotate-certificates="false" Jul 02 21:36:01.821090 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812399 2556 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 02 21:36:01.821090 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812402 2556 flags.go:64] FLAG: --runonce="false" Jul 02 21:36:01.821090 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812404 2556 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 02 21:36:01.821090 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812407 2556 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 02 21:36:01.821090 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812412 2556 flags.go:64] FLAG: --seccomp-default="false" Jul 02 21:36:01.821090 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812416 2556 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 02 21:36:01.821090 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812422 2556 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 02 21:36:01.821090 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812426 2556 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 02 21:36:01.821090 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812430 2556 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 02 21:36:01.821090 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812433 2556 flags.go:64] FLAG: --storage-driver-password="root" Jul 02 21:36:01.821090 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812436 2556 flags.go:64] FLAG: --storage-driver-secure="false" Jul 02 21:36:01.821090 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812438 2556 flags.go:64] FLAG: --storage-driver-table="stats" Jul 02 21:36:01.821090 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812441 2556 flags.go:64] FLAG: --storage-driver-user="root" Jul 02 21:36:01.821090 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812444 2556 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 02 21:36:01.821090 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812447 2556 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 02 21:36:01.821090 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812450 2556 flags.go:64] FLAG: --system-cgroups="" Jul 02 21:36:01.821090 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812452 2556 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 02 21:36:01.821090 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812458 2556 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 02 21:36:01.821090 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812461 2556 flags.go:64] FLAG: --tls-cert-file="" Jul 02 21:36:01.821090 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812463 2556 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 02 21:36:01.821090 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812469 2556 flags.go:64] FLAG: --tls-min-version="" Jul 02 21:36:01.821090 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812471 2556 flags.go:64] FLAG: --tls-private-key-file="" Jul 02 21:36:01.821090 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812475 2556 flags.go:64] FLAG: --topology-manager-policy="none" Jul 02 21:36:01.821090 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812478 2556 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 02 21:36:01.821090 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812480 2556 flags.go:64] FLAG: --topology-manager-scope="container" Jul 02 21:36:01.821739 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812483 2556 flags.go:64] FLAG: --v="2" Jul 02 21:36:01.821739 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812487 2556 flags.go:64] FLAG: --version="false" Jul 02 21:36:01.821739 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812491 2556 flags.go:64] FLAG: --vmodule="" Jul 02 21:36:01.821739 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812494 2556 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 02 21:36:01.821739 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812498 2556 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 02 21:36:01.821739 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812622 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 21:36:01.821739 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812626 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 21:36:01.821739 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812629 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 21:36:01.821739 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812632 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 21:36:01.821739 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812635 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 21:36:01.821739 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812638 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 21:36:01.821739 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812641 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 21:36:01.821739 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812643 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 21:36:01.821739 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812647 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 21:36:01.821739 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812650 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 21:36:01.821739 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812653 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 21:36:01.821739 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812655 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 21:36:01.821739 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812658 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 21:36:01.821739 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812661 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 21:36:01.821739 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812663 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 02 21:36:01.821739 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812666 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 21:36:01.822248 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812669 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 21:36:01.822248 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812671 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 21:36:01.822248 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812675 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 21:36:01.822248 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812679 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 21:36:01.822248 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812682 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 21:36:01.822248 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812685 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 21:36:01.822248 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812689 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 21:36:01.822248 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812692 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 21:36:01.822248 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812695 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 21:36:01.822248 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812698 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 21:36:01.822248 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812701 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 21:36:01.822248 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812703 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 21:36:01.822248 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812706 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 21:36:01.822248 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812708 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 21:36:01.822248 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812711 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 21:36:01.822248 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812713 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 21:36:01.822248 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812716 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 21:36:01.822248 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812718 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 21:36:01.822248 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812721 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 21:36:01.822248 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812723 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 21:36:01.822753 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812726 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 21:36:01.822753 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812728 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 21:36:01.822753 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812730 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 21:36:01.822753 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812733 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 21:36:01.822753 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812737 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 21:36:01.822753 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812739 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 21:36:01.822753 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812742 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 21:36:01.822753 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812745 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 21:36:01.822753 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812747 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 21:36:01.822753 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812750 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 21:36:01.822753 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812752 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 21:36:01.822753 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812755 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 21:36:01.822753 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812757 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 21:36:01.822753 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812760 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 21:36:01.822753 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812763 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 21:36:01.822753 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812765 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 21:36:01.822753 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812768 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 21:36:01.822753 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812770 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 21:36:01.822753 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812774 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 21:36:01.822753 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812777 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 21:36:01.823283 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812779 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 21:36:01.823283 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812782 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 21:36:01.823283 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812785 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 21:36:01.823283 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812787 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 21:36:01.823283 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812790 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 21:36:01.823283 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812792 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 21:36:01.823283 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812794 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 21:36:01.823283 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812797 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 21:36:01.823283 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812799 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 21:36:01.823283 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812802 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 21:36:01.823283 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812804 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 21:36:01.823283 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812807 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 21:36:01.823283 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812809 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 21:36:01.823283 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812811 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 21:36:01.823283 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812814 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 21:36:01.823283 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812816 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 21:36:01.823283 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812820 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 21:36:01.823283 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812823 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 21:36:01.823283 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812825 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 21:36:01.823759 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812828 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 21:36:01.823759 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812830 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 21:36:01.823759 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812834 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 21:36:01.823759 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812838 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 21:36:01.823759 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812841 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 21:36:01.823759 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812844 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 21:36:01.823759 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812847 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 21:36:01.823759 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812851 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 21:36:01.823759 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812854 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 21:36:01.823759 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812857 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 21:36:01.823759 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812859 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 21:36:01.823759 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812863 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 21:36:01.823759 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812867 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 21:36:01.823759 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812869 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 21:36:01.823759 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812872 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 21:36:01.823759 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812875 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 21:36:01.823759 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812877 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 21:36:01.823759 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812880 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 21:36:01.823759 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812883 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 21:36:01.824223 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812885 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 21:36:01.824223 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812887 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 21:36:01.824223 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812890 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 21:36:01.824223 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812892 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 21:36:01.824223 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.812895 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 21:36:01.824223 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.812903 2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 21:36:01.824223 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.820990 2556 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 02 21:36:01.824223 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.821004 2556 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 02 21:36:01.824223 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.821023 2556 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 02 21:36:01.824223 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.821028 2556 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 02 21:36:01.824223 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821084 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 21:36:01.824223 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821090 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 21:36:01.824223 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821093 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 21:36:01.824223 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821097 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 21:36:01.824223 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821100 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 21:36:01.824604 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821103 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 21:36:01.824604 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821105 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 21:36:01.824604 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821108 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 21:36:01.824604 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821111 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 21:36:01.824604 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821114 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 02 21:36:01.824604 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821116 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 21:36:01.824604 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821119 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 21:36:01.824604 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821121 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 21:36:01.824604 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821124 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 21:36:01.824604 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821127 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 21:36:01.824604 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821129 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 21:36:01.824604 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821132 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 21:36:01.824604 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821135 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 21:36:01.824604 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821137 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 21:36:01.824604 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821140 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 21:36:01.824604 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821142 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 21:36:01.824604 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821145 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 21:36:01.824604 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821148 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 21:36:01.824604 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821150 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 21:36:01.824604 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821153 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 21:36:01.825095 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821155 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 21:36:01.825095 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821158 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 21:36:01.825095 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821160 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 21:36:01.825095 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821162 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 21:36:01.825095 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821165 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 21:36:01.825095 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821169 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 21:36:01.825095 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821171 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 21:36:01.825095 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821174 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 21:36:01.825095 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821177 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 21:36:01.825095 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821179 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 21:36:01.825095 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821181 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 21:36:01.825095 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821185 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 21:36:01.825095 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821187 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 21:36:01.825095 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821189 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 21:36:01.825095 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821192 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 21:36:01.825095 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821194 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 21:36:01.825095 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821197 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 21:36:01.825095 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821199 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 21:36:01.825095 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821202 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 21:36:01.825095 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821204 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 21:36:01.825590 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821206 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 21:36:01.825590 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821209 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 21:36:01.825590 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821212 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 21:36:01.825590 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821215 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 21:36:01.825590 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821217 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 21:36:01.825590 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821220 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 21:36:01.825590 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821222 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 21:36:01.825590 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821225 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 21:36:01.825590 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821227 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 21:36:01.825590 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821229 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 21:36:01.825590 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821232 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 21:36:01.825590 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821235 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 21:36:01.825590 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821237 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 21:36:01.825590 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821240 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 21:36:01.825590 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821242 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 21:36:01.825590 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821244 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 21:36:01.825590 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821248 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 21:36:01.825590 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821250 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 21:36:01.825590 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821254 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 21:36:01.825590 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821256 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 21:36:01.826064 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821258 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 21:36:01.826064 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821261 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 21:36:01.826064 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821264 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 21:36:01.826064 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821266 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 21:36:01.826064 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821269 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 21:36:01.826064 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821271 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 21:36:01.826064 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821274 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 21:36:01.826064 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821276 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 21:36:01.826064 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821279 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 21:36:01.826064 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821282 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 21:36:01.826064 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821284 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 21:36:01.826064 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821286 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 21:36:01.826064 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821289 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 21:36:01.826064 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821292 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 21:36:01.826064 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821295 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 21:36:01.826064 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821297 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 21:36:01.826064 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821300 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 21:36:01.826064 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821303 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 21:36:01.826064 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821307 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 21:36:01.826590 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821312 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 21:36:01.826590 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821315 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 21:36:01.826590 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821318 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 21:36:01.826590 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821321 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 21:36:01.826590 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821324 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 21:36:01.826590 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821327 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 21:36:01.826590 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821329 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 21:36:01.826590 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821332 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 21:36:01.826590 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821335 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 21:36:01.826590 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821339 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 21:36:01.826590 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821341 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 21:36:01.826590 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821344 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 21:36:01.826590 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821348 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 21:36:01.826590 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821360 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 21:36:01.826590 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821363 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 21:36:01.826967 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.821368 2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 21:36:01.826967 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821480 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 21:36:01.826967 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821485 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 21:36:01.826967 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821488 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 21:36:01.826967 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821491 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 21:36:01.826967 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821493 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 21:36:01.826967 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821496 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 21:36:01.826967 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821499 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 21:36:01.826967 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821501 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 21:36:01.826967 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821504 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 21:36:01.826967 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821506 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 21:36:01.826967 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821509 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 21:36:01.826967 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821511 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 21:36:01.826967 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821530 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 21:36:01.826967 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821535 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 21:36:01.827338 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821539 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 21:36:01.827338 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821543 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 21:36:01.827338 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821547 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 21:36:01.827338 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821562 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 21:36:01.827338 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821566 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 21:36:01.827338 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821570 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 21:36:01.827338 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821574 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 21:36:01.827338 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821578 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 21:36:01.827338 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821583 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 21:36:01.827338 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821586 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 21:36:01.827338 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821588 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 21:36:01.827338 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821591 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 21:36:01.827338 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821593 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 21:36:01.827338 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821596 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 21:36:01.827338 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821599 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 21:36:01.827338 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821602 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 21:36:01.827338 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821604 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 21:36:01.827338 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821607 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 21:36:01.827338 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821609 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 21:36:01.827338 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821612 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 21:36:01.827839 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821614 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 21:36:01.827839 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821617 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 21:36:01.827839 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821619 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 21:36:01.827839 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821621 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 21:36:01.827839 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821623 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 21:36:01.827839 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821626 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 21:36:01.827839 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821628 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 21:36:01.827839 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821631 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 21:36:01.827839 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821633 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 21:36:01.827839 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821636 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 21:36:01.827839 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821638 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 21:36:01.827839 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821642 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 21:36:01.827839 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821644 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 21:36:01.827839 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821647 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 02 21:36:01.827839 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821650 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 21:36:01.827839 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821653 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 21:36:01.827839 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821655 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 21:36:01.827839 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821658 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 21:36:01.827839 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821660 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 21:36:01.827839 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821662 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 21:36:01.828333 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821665 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 21:36:01.828333 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821668 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 21:36:01.828333 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821670 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 21:36:01.828333 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821672 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 21:36:01.828333 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821675 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 21:36:01.828333 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821678 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 21:36:01.828333 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821680 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 21:36:01.828333 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821683 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 21:36:01.828333 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821685 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 21:36:01.828333 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821687 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 21:36:01.828333 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821690 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 21:36:01.828333 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821692 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 21:36:01.828333 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821695 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 21:36:01.828333 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821697 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 21:36:01.828333 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821701 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 21:36:01.828333 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821704 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 21:36:01.828333 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821707 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 21:36:01.828333 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821710 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 21:36:01.828333 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821712 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 21:36:01.828333 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821716 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 21:36:01.828820 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821719 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 21:36:01.828820 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821722 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 21:36:01.828820 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821724 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 21:36:01.828820 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821727 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 21:36:01.828820 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821731 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 21:36:01.828820 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821734 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 21:36:01.828820 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821736 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 21:36:01.828820 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821739 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 21:36:01.828820 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821742 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 21:36:01.828820 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821744 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 21:36:01.828820 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821747 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 21:36:01.828820 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821751 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 21:36:01.828820 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821755 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 21:36:01.828820 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821758 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 21:36:01.828820 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821760 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 21:36:01.828820 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821763 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 21:36:01.828820 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821766 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 21:36:01.828820 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821769 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 21:36:01.828820 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821771 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 21:36:01.829267 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821774 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 21:36:01.829267 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821776 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 21:36:01.829267 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821778 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 21:36:01.829267 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821781 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 21:36:01.829267 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821784 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 21:36:01.829267 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:01.821786 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 21:36:01.829267 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.821791 2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 21:36:01.829267 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.821886 2556 server.go:962] "Client rotation is on, will bootstrap in background" Jul 02 21:36:01.829267 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.824794 2556 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 02 21:36:01.829267 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.826201 2556 server.go:1019] "Starting client certificate rotation" Jul 02 21:36:01.829267 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.826309 2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 21:36:01.829267 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.826341 2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 21:36:01.829267 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.828651 2556 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 21:36:01.829793 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.829777 2556 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 21:36:01.839498 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.839480 2556 log.go:25] "Validated CRI v1 runtime API" Jul 02 21:36:01.843328 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.843310 2556 log.go:25] "Validated CRI v1 image API" Jul 02 21:36:01.844151 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.844137 2556 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 02 21:36:01.846734 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.846717 2556 fs.go:135] Filesystem UUIDs: map[257dff71-30f8-4667-9598-5758958ef09b:/dev/nvme0n1p3 5263f5ea-be08-4e9c-9cb8-894b06fc5184:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 02 21:36:01.846786 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.846735 2556 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 02 21:36:01.852713 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.852616 2556 manager.go:216] Machine: {Timestamp:2026-07-02 21:36:01.851157854 +0000 UTC m=+0.350861389 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3113294 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2af66808e42aa7cc5eadfd686c0cfe SystemUUID:ec2af668-08e4-2aa7-cc5e-adfd686c0cfe BootID:bb118791-0520-4b51-b392-12c2064852e6 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:fa:e0:71:ba:7b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:fa:e0:71:ba:7b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1a:c1:6a:ca:a8:a6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 02 21:36:01.852713 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.852708 2556 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 02 21:36:01.852807 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.852778 2556 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 02 21:36:01.855933 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.855900 2556 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 02 21:36:01.855996 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.855982 2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 21:36:01.856102 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.855936 2556 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-146.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 02 21:36:01.856740 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.856730 2556 topology_manager.go:138] "Creating topology manager with none policy" Jul 02 21:36:01.856770 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.856742 2556 container_manager_linux.go:309] "Creating device plugin manager" Jul 02 21:36:01.856770 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.856749 2556 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 21:36:01.857429 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.857419 2556 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 21:36:01.858719 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.858708 2556 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 02 21:36:01.861842 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.861831 2556 state_mem.go:36] "Initialized new in-memory state store" Jul 02 21:36:01.861953 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.861943 2556 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 02 21:36:01.861989 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.861976 2556 kubelet.go:486] "Attempting to sync node with API server" Jul 02 21:36:01.861989 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.861988 2556 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 02 21:36:01.862083 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.862018 2556 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 02 21:36:01.862083 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.862031 2556 kubelet.go:398] "Adding apiserver pod source" Jul 02 21:36:01.862083 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.862042 2556 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 02 21:36:01.862451 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.862421 2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 21:36:01.862451 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.862439 2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 21:36:01.867388 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.867368 2556 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 02 21:36:01.869263 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.869247 2556 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 02 21:36:01.869350 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.869286 2556 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 02 21:36:01.870432 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.870418 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 02 21:36:01.870491 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.870436 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 02 21:36:01.870491 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.870442 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 02 21:36:01.870491 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.870448 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 02 21:36:01.870491 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.870457 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 02 21:36:01.870491 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.870462 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 02 21:36:01.870491 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.870469 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 02 21:36:01.870491 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.870474 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 02 21:36:01.870491 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.870480 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 02 21:36:01.870491 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.870485 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 02 21:36:01.870491 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.870494 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 02 21:36:01.870766 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.870499 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 02 21:36:01.871354 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.871340 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 02 21:36:01.871392 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.871359 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 02 21:36:01.871735 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:01.871711 2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 02 21:36:01.871865 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:01.871847 2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-146.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 02 21:36:01.874553 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.874541 2556 server.go:1268] "Started kubelet" Jul 02 21:36:01.874712 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.874665 2556 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 02 21:36:01.874804 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.874738 2556 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 02 21:36:01.875225 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.875199 2556 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 02 21:36:01.875471 ip-10-0-140-146 systemd[1]: Started Kubernetes Kubelet. Jul 02 21:36:01.876262 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.876248 2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xnphh" Jul 02 21:36:01.877061 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.877049 2556 server.go:310] "Adding debug handlers to kubelet server" Jul 02 21:36:01.878654 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.878639 2556 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 02 21:36:01.880987 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.880956 2556 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-146.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 02 21:36:01.881086 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:01.881067 2556 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 02 21:36:01.881779 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:01.880857 2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-146.ec2.internal.18be96aa950bff04 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-146.ec2.internal,UID:ip-10-0-140-146.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-146.ec2.internal,},FirstTimestamp:2026-07-02 21:36:01.874493188 +0000 UTC m=+0.374196721,LastTimestamp:2026-07-02 21:36:01.874493188 +0000 UTC m=+0.374196721,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-146.ec2.internal,}" Jul 02 21:36:01.883511 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.883496 2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xnphh" Jul 02 21:36:01.883841 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.883818 2556 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 02 21:36:01.883841 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.883830 2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 02 21:36:01.884021 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:01.883887 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-146.ec2.internal\" not found" Jul 02 21:36:01.884813 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.884682 2556 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 02 21:36:01.884813 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.884694 2556 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 02 21:36:01.884813 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.884711 2556 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 02 21:36:01.884813 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.884783 2556 factory.go:55] Registering systemd factory Jul 02 21:36:01.884813 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.884799 2556 factory.go:223] Registration of the systemd container factory successfully Jul 02 21:36:01.885016 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.884823 2556 reconstruct.go:97] "Volume reconstruction finished" Jul 02 21:36:01.885016 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.884831 2556 reconciler.go:29] "Reconciler: start to sync state" Jul 02 21:36:01.885016 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.885001 2556 factory.go:153] Registering CRI-O factory Jul 02 21:36:01.885016 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.885011 2556 factory.go:223] Registration of the crio container factory successfully Jul 02 21:36:01.885219 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.885084 2556 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 02 21:36:01.885219 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.885107 2556 factory.go:103] Registering Raw factory Jul 02 21:36:01.885219 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.885121 2556 manager.go:1196] Started watching for new ooms in manager Jul 02 21:36:01.885587 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.885574 2556 manager.go:349] Starting recovery of all containers Jul 02 21:36:01.888336 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:01.888165 2556 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 02 21:36:01.888422 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:01.888181 2556 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-146.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 02 21:36:01.896111 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.896084 2556 manager.go:354] Recovery completed Jul 02 21:36:01.898186 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:01.898168 2556 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 02 21:36:01.900960 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.900948 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 21:36:01.902864 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.902851 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-146.ec2.internal" event="NodeHasSufficientMemory" Jul 02 21:36:01.902917 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.902876 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-146.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 21:36:01.902917 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.902886 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-146.ec2.internal" event="NodeHasSufficientPID" Jul 02 21:36:01.903338 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.903324 2556 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 02 21:36:01.903372 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.903335 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 21:36:01.903372 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.903350 2556 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 02 21:36:01.903372 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.903368 2556 state_mem.go:36] "Initialized new in-memory state store" Jul 02 21:36:01.904013 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.904000 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-146.ec2.internal" event="NodeHasSufficientMemory" Jul 02 21:36:01.904063 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.904025 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-146.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 21:36:01.904063 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.904036 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-146.ec2.internal" event="NodeHasSufficientPID" Jul 02 21:36:01.904653 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.904636 2556 policy_none.go:49] "None policy: Start" Jul 02 21:36:01.904653 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.904656 2556 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 02 21:36:01.904816 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.904669 2556 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 02 21:36:01.904869 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:01.904774 2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-146.ec2.internal.18be96aa96bce839 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-146.ec2.internal,UID:ip-10-0-140-146.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-146.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-146.ec2.internal,},FirstTimestamp:2026-07-02 21:36:01.902864441 +0000 UTC m=+0.402567975,LastTimestamp:2026-07-02 21:36:01.902864441 +0000 UTC m=+0.402567975,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-146.ec2.internal,}" Jul 02 21:36:01.905811 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.905800 2556 policy_none.go:47] "Start" Jul 02 21:36:01.948087 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.948071 2556 manager.go:337] "Starting Device Plugin manager" Jul 02 21:36:01.948151 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:01.948110 2556 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 02 21:36:01.948151 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.948123 2556 server.go:87] "Starting device plugin registration server" Jul 02 21:36:01.948360 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.948341 2556 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 02 21:36:01.948449 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.948358 2556 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 02 21:36:01.948449 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.948441 2556 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 02 21:36:01.948552 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.948538 2556 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 02 21:36:01.948552 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:01.948548 2556 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 02 21:36:01.948993 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:01.948972 2556 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 02 21:36:01.949071 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:01.949027 2556 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-146.ec2.internal\" not found" Jul 02 21:36:02.028155 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.028132 2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 02 21:36:02.029481 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.029465 2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 02 21:36:02.029555 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.029500 2556 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 02 21:36:02.029555 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.029535 2556 kubelet.go:2448] "Starting kubelet main sync loop" Jul 02 21:36:02.029642 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:02.029565 2556 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 02 21:36:02.031674 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.031651 2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 21:36:02.048699 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.048684 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 21:36:02.049534 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.049505 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-146.ec2.internal" event="NodeHasSufficientMemory" Jul 02 21:36:02.049612 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.049577 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-146.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 21:36:02.049612 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.049587 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-146.ec2.internal" event="NodeHasSufficientPID" Jul 02 21:36:02.049612 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.049608 2556 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-146.ec2.internal" Jul 02 21:36:02.057656 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.057623 2556 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-146.ec2.internal" Jul 02 21:36:02.057656 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:02.057643 2556 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-146.ec2.internal\": node \"ip-10-0-140-146.ec2.internal\" not found" Jul 02 21:36:02.066318 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:02.066304 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-146.ec2.internal\" not found" Jul 02 21:36:02.130224 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.130205 2556 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-140-146.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-146.ec2.internal"] Jul 02 21:36:02.130275 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.130264 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 21:36:02.130988 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.130974 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-146.ec2.internal" event="NodeHasSufficientMemory" Jul 02 21:36:02.131046 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.130998 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-146.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 21:36:02.131046 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.131009 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-146.ec2.internal" event="NodeHasSufficientPID" Jul 02 21:36:02.132680 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.132668 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 21:36:02.132779 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.132767 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-146.ec2.internal" Jul 02 21:36:02.132812 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.132792 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 21:36:02.133270 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.133256 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-146.ec2.internal" event="NodeHasSufficientMemory" Jul 02 21:36:02.133330 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.133256 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-146.ec2.internal" event="NodeHasSufficientMemory" Jul 02 21:36:02.133330 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.133297 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-146.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 21:36:02.133330 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.133305 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-146.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 21:36:02.133330 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.133309 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-146.ec2.internal" event="NodeHasSufficientPID" Jul 02 21:36:02.133330 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.133315 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-146.ec2.internal" event="NodeHasSufficientPID" Jul 02 21:36:02.134534 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.134504 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-146.ec2.internal" Jul 02 21:36:02.134602 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.134551 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 21:36:02.135130 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.135115 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-146.ec2.internal" event="NodeHasSufficientMemory" Jul 02 21:36:02.135221 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.135139 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-146.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 21:36:02.135221 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.135154 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-146.ec2.internal" event="NodeHasSufficientPID" Jul 02 21:36:02.156072 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:02.156053 2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-146.ec2.internal\" not found" node="ip-10-0-140-146.ec2.internal" Jul 02 21:36:02.160294 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:02.160281 2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-146.ec2.internal\" not found" node="ip-10-0-140-146.ec2.internal" Jul 02 21:36:02.166527 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:02.166502 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-146.ec2.internal\" not found" Jul 02 21:36:02.187242 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.187223 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8861d880e93a504d75b18448f12e97fd-config\") pod \"kube-apiserver-proxy-ip-10-0-140-146.ec2.internal\" (UID: \"8861d880e93a504d75b18448f12e97fd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-146.ec2.internal" Jul 02 21:36:02.187296 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.187246 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0670fa63672b3eeec21da85e068d962b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-146.ec2.internal\" (UID: \"0670fa63672b3eeec21da85e068d962b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-146.ec2.internal" Jul 02 21:36:02.187296 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.187263 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0670fa63672b3eeec21da85e068d962b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-146.ec2.internal\" (UID: \"0670fa63672b3eeec21da85e068d962b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-146.ec2.internal" Jul 02 21:36:02.267095 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:02.267071 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-146.ec2.internal\" not found" Jul 02 21:36:02.287800 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.287778 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8861d880e93a504d75b18448f12e97fd-config\") pod \"kube-apiserver-proxy-ip-10-0-140-146.ec2.internal\" (UID: \"8861d880e93a504d75b18448f12e97fd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-146.ec2.internal" Jul 02 21:36:02.287856 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.287805 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0670fa63672b3eeec21da85e068d962b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-146.ec2.internal\" (UID: \"0670fa63672b3eeec21da85e068d962b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-146.ec2.internal" Jul 02 21:36:02.287856 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.287822 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0670fa63672b3eeec21da85e068d962b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-146.ec2.internal\" (UID: \"0670fa63672b3eeec21da85e068d962b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-146.ec2.internal" Jul 02 21:36:02.287927 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.287877 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0670fa63672b3eeec21da85e068d962b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-146.ec2.internal\" (UID: \"0670fa63672b3eeec21da85e068d962b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-146.ec2.internal" Jul 02 21:36:02.287927 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.287915 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8861d880e93a504d75b18448f12e97fd-config\") pod \"kube-apiserver-proxy-ip-10-0-140-146.ec2.internal\" (UID: \"8861d880e93a504d75b18448f12e97fd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-146.ec2.internal" Jul 02 21:36:02.287997 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.287936 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0670fa63672b3eeec21da85e068d962b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-146.ec2.internal\" (UID: \"0670fa63672b3eeec21da85e068d962b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-146.ec2.internal" Jul 02 21:36:02.367185 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:02.367123 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-146.ec2.internal\" not found" Jul 02 21:36:02.459702 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.459681 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-146.ec2.internal" Jul 02 21:36:02.464773 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.464758 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-146.ec2.internal" Jul 02 21:36:02.467274 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:02.467253 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-146.ec2.internal\" not found" Jul 02 21:36:02.568015 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:02.567985 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-146.ec2.internal\" not found" Jul 02 21:36:02.668580 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:02.668536 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-146.ec2.internal\" not found" Jul 02 21:36:02.769099 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:02.769070 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-146.ec2.internal\" not found" Jul 02 21:36:02.788953 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.788936 2556 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 21:36:02.825835 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.825814 2556 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 02 21:36:02.826480 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.825914 2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 02 21:36:02.826480 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.825945 2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 02 21:36:02.869297 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:02.869273 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-146.ec2.internal\" not found" Jul 02 21:36:02.897589 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.897551 2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-01 21:31:03 +0000 UTC" deadline="2028-01-11 21:26:30.321954247 +0000 UTC" Jul 02 21:36:02.897589 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:02.897586 2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13391h50m27.424373299s" Jul 02 21:36:04.749693 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:04.749672 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-146.ec2.internal\" not found" Jul 02 21:36:04.795919 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:04.795900 2556 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 21:36:04.864665 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:04.864641 2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-146.ec2.internal" Jul 02 21:36:04.870818 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:04.870791 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8861d880e93a504d75b18448f12e97fd.slice/crio-c55840c87714aef610958575d504764cdef7d5dacdb819d9b2091e800358c3b3 WatchSource:0}: Error finding container c55840c87714aef610958575d504764cdef7d5dacdb819d9b2091e800358c3b3: Status 404 returned error can't find the container with id c55840c87714aef610958575d504764cdef7d5dacdb819d9b2091e800358c3b3 Jul 02 21:36:04.871425 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:04.871401 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0670fa63672b3eeec21da85e068d962b.slice/crio-8180edf55a51ffad6d8209b527bf981dcb9c0e5ea569e30bf4100284226b99a7 WatchSource:0}: Error finding container 8180edf55a51ffad6d8209b527bf981dcb9c0e5ea569e30bf4100284226b99a7: Status 404 returned error can't find the container with id 8180edf55a51ffad6d8209b527bf981dcb9c0e5ea569e30bf4100284226b99a7 Jul 02 21:36:04.874614 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:04.874594 2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 21:36:04.875435 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:04.875423 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 21:36:04.876012 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:04.875996 2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-146.ec2.internal" Jul 02 21:36:04.884812 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:04.884796 2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 21:36:05.095778 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.095755 2556 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 21:36:05.644033 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.643997 2556 apiserver.go:52] "Watching apiserver" Jul 02 21:36:05.652037 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.652014 2556 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 02 21:36:05.652487 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.652469 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-mtlf9","kube-system/kube-apiserver-proxy-ip-10-0-140-146.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f4vzj","openshift-cluster-node-tuning-operator/tuned-ntfv6","openshift-image-registry/node-ca-82ctv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-146.ec2.internal","openshift-multus/multus-additional-cni-plugins-qmwz4","openshift-multus/multus-cdxjn","kube-system/global-pull-secret-syncer-bg9p7","openshift-dns/node-resolver-jgmz2","openshift-multus/network-metrics-daemon-zp6sf","openshift-network-diagnostics/network-check-target-zkjd5","openshift-network-operator/iptables-alerter-5hc6v","openshift-ovn-kubernetes/ovnkube-node-s8jx8"] Jul 02 21:36:05.654249 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.654227 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qmwz4" Jul 02 21:36:05.655721 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.655693 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5hc6v" Jul 02 21:36:05.656888 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.656847 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 02 21:36:05.656980 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.656922 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 02 21:36:05.657176 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.657159 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 02 21:36:05.657523 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.657260 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f4vzj" Jul 02 21:36:05.657523 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.657373 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 02 21:36:05.657523 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.657385 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-lv45z\"" Jul 02 21:36:05.657735 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.657717 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 02 21:36:05.658489 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.658470 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 02 21:36:05.658604 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.658565 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-h2skp\"" Jul 02 21:36:05.658673 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.658470 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 02 21:36:05.659184 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.658952 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.659268 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.659235 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 02 21:36:05.660011 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.659992 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 02 21:36:05.660344 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.660303 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 02 21:36:05.660758 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.660738 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 02 21:36:05.660824 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.660796 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mjpkf\"" Jul 02 21:36:05.661044 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.661031 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 02 21:36:05.661105 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.661046 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-82ctv" Jul 02 21:36:05.661345 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.661324 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cbgxj\"" Jul 02 21:36:05.661420 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.661328 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 02 21:36:05.662509 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.662423 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.663326 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.663297 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 02 21:36:05.663528 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.663513 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 02 21:36:05.663750 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.663693 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 02 21:36:05.663883 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.663845 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jh89m\"" Jul 02 21:36:05.664371 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.664355 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg9p7" Jul 02 21:36:05.664462 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.664374 2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 02 21:36:05.664462 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:05.664440 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bg9p7" podUID="27a3595e-6df7-4999-8257-5c111b074330" Jul 02 21:36:05.664683 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.664667 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 02 21:36:05.665116 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.665099 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 02 21:36:05.666162 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.666138 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 02 21:36:05.666410 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.666392 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mtlf9" Jul 02 21:36:05.666479 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.666428 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 02 21:36:05.666725 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.666701 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-wr2tc\"" Jul 02 21:36:05.666809 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.666792 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 02 21:36:05.667339 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.667324 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 02 21:36:05.669098 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.669078 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 02 21:36:05.669545 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.669510 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.670343 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.670012 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 02 21:36:05.670586 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.670568 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-wnsz7\"" Jul 02 21:36:05.672115 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.671836 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jgmz2" Jul 02 21:36:05.672115 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.672102 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-w5cf9\"" Jul 02 21:36:05.672376 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.672353 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 02 21:36:05.673662 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.673401 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zp6sf" Jul 02 21:36:05.673662 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:05.673453 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zp6sf" podUID="0bc3fefa-dff5-4181-a239-c6573b8b4356" Jul 02 21:36:05.674313 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.674293 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 02 21:36:05.674429 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.674413 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vt796\"" Jul 02 21:36:05.674489 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.674432 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 02 21:36:05.674685 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.674668 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkjd5" Jul 02 21:36:05.674795 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:05.674722 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkjd5" podUID="298fd481-12c6-4869-a4d4-9b17c492706a" Jul 02 21:36:05.674976 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.674816 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-node-log\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.674976 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.674848 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t86k\" (UniqueName: \"kubernetes.io/projected/43a2d69d-8f93-4618-a04d-044c71f7e00b-kube-api-access-6t86k\") pod \"iptables-alerter-5hc6v\" (UID: \"43a2d69d-8f93-4618-a04d-044c71f7e00b\") " pod="openshift-network-operator/iptables-alerter-5hc6v" Jul 02 21:36:05.674976 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.674893 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cb5991b9-b1ea-46a5-9398-fe121b19bb4f-socket-dir\") pod \"aws-ebs-csi-driver-node-f4vzj\" (UID: \"cb5991b9-b1ea-46a5-9398-fe121b19bb4f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f4vzj" Jul 02 21:36:05.674976 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.674916 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/475d8de2-f070-4c1e-850e-89fecc63ecfe-etc-tuned\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.674976 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.674947 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cb5991b9-b1ea-46a5-9398-fe121b19bb4f-sys-fs\") pod \"aws-ebs-csi-driver-node-f4vzj\" (UID: \"cb5991b9-b1ea-46a5-9398-fe121b19bb4f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f4vzj" Jul 02 21:36:05.674976 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.674970 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/475d8de2-f070-4c1e-850e-89fecc63ecfe-etc-kubernetes\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.675310 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.674991 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/475d8de2-f070-4c1e-850e-89fecc63ecfe-etc-systemd\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.675310 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675010 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/475d8de2-f070-4c1e-850e-89fecc63ecfe-run\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.675310 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675034 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-var-lib-openvswitch\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.675310 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675057 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cb5991b9-b1ea-46a5-9398-fe121b19bb4f-registration-dir\") pod \"aws-ebs-csi-driver-node-f4vzj\" (UID: \"cb5991b9-b1ea-46a5-9398-fe121b19bb4f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f4vzj" Jul 02 21:36:05.675310 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675079 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/13dcc457-ac6d-416b-82e8-191fa49d0349-konnectivity-ca\") pod \"konnectivity-agent-mtlf9\" (UID: \"13dcc457-ac6d-416b-82e8-191fa49d0349\") " pod="kube-system/konnectivity-agent-mtlf9" Jul 02 21:36:05.675310 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675102 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eb1cecd6-d74b-4f21-bb0a-02f534221301-system-cni-dir\") pod \"multus-additional-cni-plugins-qmwz4\" (UID: \"eb1cecd6-d74b-4f21-bb0a-02f534221301\") " pod="openshift-multus/multus-additional-cni-plugins-qmwz4" Jul 02 21:36:05.675310 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675127 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cb5991b9-b1ea-46a5-9398-fe121b19bb4f-etc-selinux\") pod \"aws-ebs-csi-driver-node-f4vzj\" (UID: \"cb5991b9-b1ea-46a5-9398-fe121b19bb4f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f4vzj" Jul 02 21:36:05.675310 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675149 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-log-socket\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.675310 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675171 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-host-cni-bin\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.675310 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675196 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/eb1cecd6-d74b-4f21-bb0a-02f534221301-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qmwz4\" (UID: \"eb1cecd6-d74b-4f21-bb0a-02f534221301\") " pod="openshift-multus/multus-additional-cni-plugins-qmwz4" Jul 02 21:36:05.675310 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675221 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-host-run-ovn-kubernetes\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.675310 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675254 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eb1cecd6-d74b-4f21-bb0a-02f534221301-os-release\") pod \"multus-additional-cni-plugins-qmwz4\" (UID: \"eb1cecd6-d74b-4f21-bb0a-02f534221301\") " pod="openshift-multus/multus-additional-cni-plugins-qmwz4" Jul 02 21:36:05.675310 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675289 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eb1cecd6-d74b-4f21-bb0a-02f534221301-cni-binary-copy\") pod \"multus-additional-cni-plugins-qmwz4\" (UID: \"eb1cecd6-d74b-4f21-bb0a-02f534221301\") " pod="openshift-multus/multus-additional-cni-plugins-qmwz4" Jul 02 21:36:05.675310 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675315 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cb5991b9-b1ea-46a5-9398-fe121b19bb4f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-f4vzj\" (UID: \"cb5991b9-b1ea-46a5-9398-fe121b19bb4f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f4vzj" Jul 02 21:36:05.675973 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675338 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98czv\" (UniqueName: \"kubernetes.io/projected/cb5991b9-b1ea-46a5-9398-fe121b19bb4f-kube-api-access-98czv\") pod \"aws-ebs-csi-driver-node-f4vzj\" (UID: \"cb5991b9-b1ea-46a5-9398-fe121b19bb4f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f4vzj" Jul 02 21:36:05.675973 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675361 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/475d8de2-f070-4c1e-850e-89fecc63ecfe-host\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.675973 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675382 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7ts9\" (UniqueName: \"kubernetes.io/projected/475d8de2-f070-4c1e-850e-89fecc63ecfe-kube-api-access-g7ts9\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.675973 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675405 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-run-systemd\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.675973 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675436 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7a32947e-1a4e-4138-8367-6def69748311-env-overrides\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.675973 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675459 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7a32947e-1a4e-4138-8367-6def69748311-ovn-node-metrics-cert\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.675973 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675481 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7a32947e-1a4e-4138-8367-6def69748311-ovnkube-script-lib\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.675973 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675508 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/475d8de2-f070-4c1e-850e-89fecc63ecfe-sys\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.675973 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675531 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-host-cni-netd\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.675973 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675566 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/13dcc457-ac6d-416b-82e8-191fa49d0349-agent-certs\") pod \"konnectivity-agent-mtlf9\" (UID: \"13dcc457-ac6d-416b-82e8-191fa49d0349\") " pod="kube-system/konnectivity-agent-mtlf9" Jul 02 21:36:05.675973 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675591 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/baf560bd-eaae-468e-bc1c-a007fe68bded-host\") pod \"node-ca-82ctv\" (UID: \"baf560bd-eaae-468e-bc1c-a007fe68bded\") " pod="openshift-image-registry/node-ca-82ctv" Jul 02 21:36:05.675973 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675627 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7a32947e-1a4e-4138-8367-6def69748311-ovnkube-config\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.675973 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675664 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/475d8de2-f070-4c1e-850e-89fecc63ecfe-etc-sysctl-d\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.675973 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675703 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/475d8de2-f070-4c1e-850e-89fecc63ecfe-lib-modules\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.675973 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675725 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-etc-openvswitch\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.675973 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675748 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/27a3595e-6df7-4999-8257-5c111b074330-dbus\") pod \"global-pull-secret-syncer-bg9p7\" (UID: \"27a3595e-6df7-4999-8257-5c111b074330\") " pod="kube-system/global-pull-secret-syncer-bg9p7" Jul 02 21:36:05.676411 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675773 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eb1cecd6-d74b-4f21-bb0a-02f534221301-cnibin\") pod \"multus-additional-cni-plugins-qmwz4\" (UID: \"eb1cecd6-d74b-4f21-bb0a-02f534221301\") " pod="openshift-multus/multus-additional-cni-plugins-qmwz4" Jul 02 21:36:05.676411 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675790 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/eb1cecd6-d74b-4f21-bb0a-02f534221301-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qmwz4\" (UID: \"eb1cecd6-d74b-4f21-bb0a-02f534221301\") " pod="openshift-multus/multus-additional-cni-plugins-qmwz4" Jul 02 21:36:05.676411 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675807 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/475d8de2-f070-4c1e-850e-89fecc63ecfe-etc-modprobe-d\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.676411 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675830 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/475d8de2-f070-4c1e-850e-89fecc63ecfe-tmp\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.676411 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675863 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmbzg\" (UniqueName: \"kubernetes.io/projected/baf560bd-eaae-468e-bc1c-a007fe68bded-kube-api-access-cmbzg\") pod \"node-ca-82ctv\" (UID: \"baf560bd-eaae-468e-bc1c-a007fe68bded\") " pod="openshift-image-registry/node-ca-82ctv" Jul 02 21:36:05.676411 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675916 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-host-slash\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.676411 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675943 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-run-openvswitch\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.676411 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675966 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.676411 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675981 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/475d8de2-f070-4c1e-850e-89fecc63ecfe-etc-sysconfig\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.676411 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.675997 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/475d8de2-f070-4c1e-850e-89fecc63ecfe-etc-sysctl-conf\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.676411 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.676024 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/475d8de2-f070-4c1e-850e-89fecc63ecfe-var-lib-kubelet\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.676411 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.676054 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t52kg\" (UniqueName: \"kubernetes.io/projected/7a32947e-1a4e-4138-8367-6def69748311-kube-api-access-t52kg\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.676411 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.676071 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/27a3595e-6df7-4999-8257-5c111b074330-original-pull-secret\") pod \"global-pull-secret-syncer-bg9p7\" (UID: \"27a3595e-6df7-4999-8257-5c111b074330\") " pod="kube-system/global-pull-secret-syncer-bg9p7" Jul 02 21:36:05.676411 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.676086 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-host-run-netns\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.676411 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.676106 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-run-ovn\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.676411 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.676132 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/27a3595e-6df7-4999-8257-5c111b074330-kubelet-config\") pod \"global-pull-secret-syncer-bg9p7\" (UID: \"27a3595e-6df7-4999-8257-5c111b074330\") " pod="kube-system/global-pull-secret-syncer-bg9p7" Jul 02 21:36:05.676974 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.676162 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/43a2d69d-8f93-4618-a04d-044c71f7e00b-iptables-alerter-script\") pod \"iptables-alerter-5hc6v\" (UID: \"43a2d69d-8f93-4618-a04d-044c71f7e00b\") " pod="openshift-network-operator/iptables-alerter-5hc6v" Jul 02 21:36:05.676974 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.676200 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cb5991b9-b1ea-46a5-9398-fe121b19bb4f-device-dir\") pod \"aws-ebs-csi-driver-node-f4vzj\" (UID: \"cb5991b9-b1ea-46a5-9398-fe121b19bb4f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f4vzj" Jul 02 21:36:05.676974 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.676229 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/baf560bd-eaae-468e-bc1c-a007fe68bded-serviceca\") pod \"node-ca-82ctv\" (UID: \"baf560bd-eaae-468e-bc1c-a007fe68bded\") " pod="openshift-image-registry/node-ca-82ctv" Jul 02 21:36:05.676974 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.676254 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-host-kubelet\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.676974 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.676274 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-systemd-units\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.676974 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.676296 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eb1cecd6-d74b-4f21-bb0a-02f534221301-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qmwz4\" (UID: \"eb1cecd6-d74b-4f21-bb0a-02f534221301\") " pod="openshift-multus/multus-additional-cni-plugins-qmwz4" Jul 02 21:36:05.676974 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.676328 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h56r7\" (UniqueName: \"kubernetes.io/projected/eb1cecd6-d74b-4f21-bb0a-02f534221301-kube-api-access-h56r7\") pod \"multus-additional-cni-plugins-qmwz4\" (UID: \"eb1cecd6-d74b-4f21-bb0a-02f534221301\") " pod="openshift-multus/multus-additional-cni-plugins-qmwz4" Jul 02 21:36:05.676974 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.676371 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/43a2d69d-8f93-4618-a04d-044c71f7e00b-host-slash\") pod \"iptables-alerter-5hc6v\" (UID: \"43a2d69d-8f93-4618-a04d-044c71f7e00b\") " pod="openshift-network-operator/iptables-alerter-5hc6v" Jul 02 21:36:05.681326 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.681303 2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 21:36:05.699252 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.699233 2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 21:36:05.707240 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.707206 2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-h4vhp" Jul 02 21:36:05.716981 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.716966 2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-h4vhp" Jul 02 21:36:05.765771 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.765749 2556 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 02 21:36:05.777176 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.777152 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/475d8de2-f070-4c1e-850e-89fecc63ecfe-etc-sysconfig\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.777302 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.777184 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/475d8de2-f070-4c1e-850e-89fecc63ecfe-etc-sysctl-conf\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.777302 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.777204 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/475d8de2-f070-4c1e-850e-89fecc63ecfe-var-lib-kubelet\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.777302 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.777265 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/475d8de2-f070-4c1e-850e-89fecc63ecfe-var-lib-kubelet\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.777302 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.777268 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t52kg\" (UniqueName: \"kubernetes.io/projected/7a32947e-1a4e-4138-8367-6def69748311-kube-api-access-t52kg\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.777302 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.777272 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/475d8de2-f070-4c1e-850e-89fecc63ecfe-etc-sysconfig\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.777302 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.777297 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/27a3595e-6df7-4999-8257-5c111b074330-original-pull-secret\") pod \"global-pull-secret-syncer-bg9p7\" (UID: \"27a3595e-6df7-4999-8257-5c111b074330\") " pod="kube-system/global-pull-secret-syncer-bg9p7" Jul 02 21:36:05.777652 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.777325 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzhpf\" (UniqueName: \"kubernetes.io/projected/222d3332-3472-479e-a8e2-8d8d9c702a9b-kube-api-access-fzhpf\") pod \"node-resolver-jgmz2\" (UID: \"222d3332-3472-479e-a8e2-8d8d9c702a9b\") " pod="openshift-dns/node-resolver-jgmz2" Jul 02 21:36:05.777652 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.777342 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/475d8de2-f070-4c1e-850e-89fecc63ecfe-etc-sysctl-conf\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.777652 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.777348 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgwrx\" (UniqueName: \"kubernetes.io/projected/0bc3fefa-dff5-4181-a239-c6573b8b4356-kube-api-access-xgwrx\") pod \"network-metrics-daemon-zp6sf\" (UID: \"0bc3fefa-dff5-4181-a239-c6573b8b4356\") " pod="openshift-multus/network-metrics-daemon-zp6sf" Jul 02 21:36:05.777652 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:05.777391 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 21:36:05.777652 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.777401 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-host-run-netns\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.777652 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.777429 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-run-ovn\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.777652 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:05.777452 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27a3595e-6df7-4999-8257-5c111b074330-original-pull-secret podName:27a3595e-6df7-4999-8257-5c111b074330 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:06.277431255 +0000 UTC m=+2.996780773 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/27a3595e-6df7-4999-8257-5c111b074330-original-pull-secret") pod "global-pull-secret-syncer-bg9p7" (UID: "27a3595e-6df7-4999-8257-5c111b074330") : object "kube-system"/"original-pull-secret" not registered Jul 02 21:36:05.777652 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.777472 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-host-run-netns\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.777652 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.777486 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/27a3595e-6df7-4999-8257-5c111b074330-kubelet-config\") pod \"global-pull-secret-syncer-bg9p7\" (UID: \"27a3595e-6df7-4999-8257-5c111b074330\") " pod="kube-system/global-pull-secret-syncer-bg9p7" Jul 02 21:36:05.777652 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.777492 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-run-ovn\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.777652 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.777515 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/43a2d69d-8f93-4618-a04d-044c71f7e00b-iptables-alerter-script\") pod \"iptables-alerter-5hc6v\" (UID: \"43a2d69d-8f93-4618-a04d-044c71f7e00b\") " pod="openshift-network-operator/iptables-alerter-5hc6v" Jul 02 21:36:05.777652 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.777540 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cb5991b9-b1ea-46a5-9398-fe121b19bb4f-device-dir\") pod \"aws-ebs-csi-driver-node-f4vzj\" (UID: \"cb5991b9-b1ea-46a5-9398-fe121b19bb4f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f4vzj" Jul 02 21:36:05.777652 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.777544 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/27a3595e-6df7-4999-8257-5c111b074330-kubelet-config\") pod \"global-pull-secret-syncer-bg9p7\" (UID: \"27a3595e-6df7-4999-8257-5c111b074330\") " pod="kube-system/global-pull-secret-syncer-bg9p7" Jul 02 21:36:05.777652 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.777559 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/baf560bd-eaae-468e-bc1c-a007fe68bded-serviceca\") pod \"node-ca-82ctv\" (UID: \"baf560bd-eaae-468e-bc1c-a007fe68bded\") " pod="openshift-image-registry/node-ca-82ctv" Jul 02 21:36:05.777652 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.777575 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-host-kubelet\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.777652 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.777597 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-systemd-units\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.777652 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.777625 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bc3fefa-dff5-4181-a239-c6573b8b4356-metrics-certs\") pod \"network-metrics-daemon-zp6sf\" (UID: \"0bc3fefa-dff5-4181-a239-c6573b8b4356\") " pod="openshift-multus/network-metrics-daemon-zp6sf" Jul 02 21:36:05.778451 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.777623 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cb5991b9-b1ea-46a5-9398-fe121b19bb4f-device-dir\") pod \"aws-ebs-csi-driver-node-f4vzj\" (UID: \"cb5991b9-b1ea-46a5-9398-fe121b19bb4f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f4vzj" Jul 02 21:36:05.778451 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.777651 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eb1cecd6-d74b-4f21-bb0a-02f534221301-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qmwz4\" (UID: \"eb1cecd6-d74b-4f21-bb0a-02f534221301\") " pod="openshift-multus/multus-additional-cni-plugins-qmwz4" Jul 02 21:36:05.778451 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.777658 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-host-kubelet\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.778451 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.777670 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-systemd-units\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.778451 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.777677 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h56r7\" (UniqueName: \"kubernetes.io/projected/eb1cecd6-d74b-4f21-bb0a-02f534221301-kube-api-access-h56r7\") pod \"multus-additional-cni-plugins-qmwz4\" (UID: \"eb1cecd6-d74b-4f21-bb0a-02f534221301\") " pod="openshift-multus/multus-additional-cni-plugins-qmwz4" Jul 02 21:36:05.778451 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.777803 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/43a2d69d-8f93-4618-a04d-044c71f7e00b-host-slash\") pod \"iptables-alerter-5hc6v\" (UID: \"43a2d69d-8f93-4618-a04d-044c71f7e00b\") " pod="openshift-network-operator/iptables-alerter-5hc6v" Jul 02 21:36:05.778451 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.777827 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-node-log\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.778451 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.777843 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-host-run-netns\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.778451 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.777858 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-multus-conf-dir\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.778451 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.777886 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eb1cecd6-d74b-4f21-bb0a-02f534221301-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qmwz4\" (UID: \"eb1cecd6-d74b-4f21-bb0a-02f534221301\") " pod="openshift-multus/multus-additional-cni-plugins-qmwz4" Jul 02 21:36:05.778451 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.777915 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-node-log\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.778451 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.777966 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-host-run-multus-certs\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.778451 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.777996 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6t86k\" (UniqueName: \"kubernetes.io/projected/43a2d69d-8f93-4618-a04d-044c71f7e00b-kube-api-access-6t86k\") pod \"iptables-alerter-5hc6v\" (UID: \"43a2d69d-8f93-4618-a04d-044c71f7e00b\") " pod="openshift-network-operator/iptables-alerter-5hc6v" Jul 02 21:36:05.778451 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778005 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/43a2d69d-8f93-4618-a04d-044c71f7e00b-host-slash\") pod \"iptables-alerter-5hc6v\" (UID: \"43a2d69d-8f93-4618-a04d-044c71f7e00b\") " pod="openshift-network-operator/iptables-alerter-5hc6v" Jul 02 21:36:05.778451 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778072 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/baf560bd-eaae-468e-bc1c-a007fe68bded-serviceca\") pod \"node-ca-82ctv\" (UID: \"baf560bd-eaae-468e-bc1c-a007fe68bded\") " pod="openshift-image-registry/node-ca-82ctv" Jul 02 21:36:05.778451 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778072 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cb5991b9-b1ea-46a5-9398-fe121b19bb4f-socket-dir\") pod \"aws-ebs-csi-driver-node-f4vzj\" (UID: \"cb5991b9-b1ea-46a5-9398-fe121b19bb4f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f4vzj" Jul 02 21:36:05.778451 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778132 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/475d8de2-f070-4c1e-850e-89fecc63ecfe-etc-tuned\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.779311 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778150 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cb5991b9-b1ea-46a5-9398-fe121b19bb4f-socket-dir\") pod \"aws-ebs-csi-driver-node-f4vzj\" (UID: \"cb5991b9-b1ea-46a5-9398-fe121b19bb4f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f4vzj" Jul 02 21:36:05.779311 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778164 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-host-var-lib-kubelet\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.779311 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778165 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/43a2d69d-8f93-4618-a04d-044c71f7e00b-iptables-alerter-script\") pod \"iptables-alerter-5hc6v\" (UID: \"43a2d69d-8f93-4618-a04d-044c71f7e00b\") " pod="openshift-network-operator/iptables-alerter-5hc6v" Jul 02 21:36:05.779311 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778196 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-etc-kubernetes\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.779311 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778228 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cb5991b9-b1ea-46a5-9398-fe121b19bb4f-sys-fs\") pod \"aws-ebs-csi-driver-node-f4vzj\" (UID: \"cb5991b9-b1ea-46a5-9398-fe121b19bb4f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f4vzj" Jul 02 21:36:05.779311 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778268 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/475d8de2-f070-4c1e-850e-89fecc63ecfe-etc-kubernetes\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.779311 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778293 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cb5991b9-b1ea-46a5-9398-fe121b19bb4f-sys-fs\") pod \"aws-ebs-csi-driver-node-f4vzj\" (UID: \"cb5991b9-b1ea-46a5-9398-fe121b19bb4f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f4vzj" Jul 02 21:36:05.779311 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778300 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/475d8de2-f070-4c1e-850e-89fecc63ecfe-etc-systemd\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.779311 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778315 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/475d8de2-f070-4c1e-850e-89fecc63ecfe-etc-kubernetes\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.779311 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778332 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/475d8de2-f070-4c1e-850e-89fecc63ecfe-run\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.779311 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778346 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/475d8de2-f070-4c1e-850e-89fecc63ecfe-etc-systemd\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.779311 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778354 2556 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 02 21:36:05.779311 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778394 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/475d8de2-f070-4c1e-850e-89fecc63ecfe-run\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.779311 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778360 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-var-lib-openvswitch\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.779311 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778394 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-var-lib-openvswitch\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.779311 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778432 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-system-cni-dir\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.779311 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778468 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/222d3332-3472-479e-a8e2-8d8d9c702a9b-hosts-file\") pod \"node-resolver-jgmz2\" (UID: \"222d3332-3472-479e-a8e2-8d8d9c702a9b\") " pod="openshift-dns/node-resolver-jgmz2" Jul 02 21:36:05.779311 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778496 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cb5991b9-b1ea-46a5-9398-fe121b19bb4f-registration-dir\") pod \"aws-ebs-csi-driver-node-f4vzj\" (UID: \"cb5991b9-b1ea-46a5-9398-fe121b19bb4f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f4vzj" Jul 02 21:36:05.779995 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778521 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-host-var-lib-cni-multus\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.779995 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778547 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/13dcc457-ac6d-416b-82e8-191fa49d0349-konnectivity-ca\") pod \"konnectivity-agent-mtlf9\" (UID: \"13dcc457-ac6d-416b-82e8-191fa49d0349\") " pod="kube-system/konnectivity-agent-mtlf9" Jul 02 21:36:05.779995 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778564 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cb5991b9-b1ea-46a5-9398-fe121b19bb4f-registration-dir\") pod \"aws-ebs-csi-driver-node-f4vzj\" (UID: \"cb5991b9-b1ea-46a5-9398-fe121b19bb4f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f4vzj" Jul 02 21:36:05.779995 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778573 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eb1cecd6-d74b-4f21-bb0a-02f534221301-system-cni-dir\") pod \"multus-additional-cni-plugins-qmwz4\" (UID: \"eb1cecd6-d74b-4f21-bb0a-02f534221301\") " pod="openshift-multus/multus-additional-cni-plugins-qmwz4" Jul 02 21:36:05.779995 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778592 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cb5991b9-b1ea-46a5-9398-fe121b19bb4f-etc-selinux\") pod \"aws-ebs-csi-driver-node-f4vzj\" (UID: \"cb5991b9-b1ea-46a5-9398-fe121b19bb4f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f4vzj" Jul 02 21:36:05.779995 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778608 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-log-socket\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.779995 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778623 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-host-cni-bin\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.779995 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778630 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eb1cecd6-d74b-4f21-bb0a-02f534221301-system-cni-dir\") pod \"multus-additional-cni-plugins-qmwz4\" (UID: \"eb1cecd6-d74b-4f21-bb0a-02f534221301\") " pod="openshift-multus/multus-additional-cni-plugins-qmwz4" Jul 02 21:36:05.779995 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778639 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-multus-cni-dir\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.779995 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778668 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/eb1cecd6-d74b-4f21-bb0a-02f534221301-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qmwz4\" (UID: \"eb1cecd6-d74b-4f21-bb0a-02f534221301\") " pod="openshift-multus/multus-additional-cni-plugins-qmwz4" Jul 02 21:36:05.779995 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778683 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-host-cni-bin\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.779995 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778713 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-host-run-ovn-kubernetes\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.779995 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778721 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-log-socket\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.779995 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778736 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cb5991b9-b1ea-46a5-9398-fe121b19bb4f-etc-selinux\") pod \"aws-ebs-csi-driver-node-f4vzj\" (UID: \"cb5991b9-b1ea-46a5-9398-fe121b19bb4f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f4vzj" Jul 02 21:36:05.779995 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778743 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-host-run-k8s-cni-cncf-io\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.779995 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778774 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eb1cecd6-d74b-4f21-bb0a-02f534221301-os-release\") pod \"multus-additional-cni-plugins-qmwz4\" (UID: \"eb1cecd6-d74b-4f21-bb0a-02f534221301\") " pod="openshift-multus/multus-additional-cni-plugins-qmwz4" Jul 02 21:36:05.779995 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778793 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-host-run-ovn-kubernetes\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.780778 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778805 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eb1cecd6-d74b-4f21-bb0a-02f534221301-cni-binary-copy\") pod \"multus-additional-cni-plugins-qmwz4\" (UID: \"eb1cecd6-d74b-4f21-bb0a-02f534221301\") " pod="openshift-multus/multus-additional-cni-plugins-qmwz4" Jul 02 21:36:05.780778 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778911 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eb1cecd6-d74b-4f21-bb0a-02f534221301-os-release\") pod \"multus-additional-cni-plugins-qmwz4\" (UID: \"eb1cecd6-d74b-4f21-bb0a-02f534221301\") " pod="openshift-multus/multus-additional-cni-plugins-qmwz4" Jul 02 21:36:05.780778 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778955 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cb5991b9-b1ea-46a5-9398-fe121b19bb4f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-f4vzj\" (UID: \"cb5991b9-b1ea-46a5-9398-fe121b19bb4f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f4vzj" Jul 02 21:36:05.780778 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.778985 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98czv\" (UniqueName: \"kubernetes.io/projected/cb5991b9-b1ea-46a5-9398-fe121b19bb4f-kube-api-access-98czv\") pod \"aws-ebs-csi-driver-node-f4vzj\" (UID: \"cb5991b9-b1ea-46a5-9398-fe121b19bb4f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f4vzj" Jul 02 21:36:05.780778 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779012 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/475d8de2-f070-4c1e-850e-89fecc63ecfe-host\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.780778 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779038 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7ts9\" (UniqueName: \"kubernetes.io/projected/475d8de2-f070-4c1e-850e-89fecc63ecfe-kube-api-access-g7ts9\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.780778 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779067 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-run-systemd\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.780778 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779095 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7a32947e-1a4e-4138-8367-6def69748311-env-overrides\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.780778 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779123 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7a32947e-1a4e-4138-8367-6def69748311-ovn-node-metrics-cert\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.780778 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779132 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/13dcc457-ac6d-416b-82e8-191fa49d0349-konnectivity-ca\") pod \"konnectivity-agent-mtlf9\" (UID: \"13dcc457-ac6d-416b-82e8-191fa49d0349\") " pod="kube-system/konnectivity-agent-mtlf9" Jul 02 21:36:05.780778 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779155 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7a32947e-1a4e-4138-8367-6def69748311-ovnkube-script-lib\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.780778 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779180 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6117c96e-d19e-48ab-a235-187d19f296d7-cni-binary-copy\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.780778 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779207 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-host-var-lib-cni-bin\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.780778 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779231 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/475d8de2-f070-4c1e-850e-89fecc63ecfe-sys\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.780778 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779255 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-host-cni-netd\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.780778 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779271 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/eb1cecd6-d74b-4f21-bb0a-02f534221301-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qmwz4\" (UID: \"eb1cecd6-d74b-4f21-bb0a-02f534221301\") " pod="openshift-multus/multus-additional-cni-plugins-qmwz4" Jul 02 21:36:05.780778 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779277 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtfh9\" (UniqueName: \"kubernetes.io/projected/6117c96e-d19e-48ab-a235-187d19f296d7-kube-api-access-gtfh9\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.781479 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779300 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eb1cecd6-d74b-4f21-bb0a-02f534221301-cni-binary-copy\") pod \"multus-additional-cni-plugins-qmwz4\" (UID: \"eb1cecd6-d74b-4f21-bb0a-02f534221301\") " pod="openshift-multus/multus-additional-cni-plugins-qmwz4" Jul 02 21:36:05.781479 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779313 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/222d3332-3472-479e-a8e2-8d8d9c702a9b-tmp-dir\") pod \"node-resolver-jgmz2\" (UID: \"222d3332-3472-479e-a8e2-8d8d9c702a9b\") " pod="openshift-dns/node-resolver-jgmz2" Jul 02 21:36:05.781479 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779337 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cb5991b9-b1ea-46a5-9398-fe121b19bb4f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-f4vzj\" (UID: \"cb5991b9-b1ea-46a5-9398-fe121b19bb4f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f4vzj" Jul 02 21:36:05.781479 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779341 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/13dcc457-ac6d-416b-82e8-191fa49d0349-agent-certs\") pod \"konnectivity-agent-mtlf9\" (UID: \"13dcc457-ac6d-416b-82e8-191fa49d0349\") " pod="kube-system/konnectivity-agent-mtlf9" Jul 02 21:36:05.781479 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779352 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/475d8de2-f070-4c1e-850e-89fecc63ecfe-host\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.781479 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779378 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/baf560bd-eaae-468e-bc1c-a007fe68bded-host\") pod \"node-ca-82ctv\" (UID: \"baf560bd-eaae-468e-bc1c-a007fe68bded\") " pod="openshift-image-registry/node-ca-82ctv" Jul 02 21:36:05.781479 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779402 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7a32947e-1a4e-4138-8367-6def69748311-ovnkube-config\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.781479 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779420 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-run-systemd\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.781479 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779426 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-os-release\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.781479 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779448 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-hostroot\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.781479 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779472 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/475d8de2-f070-4c1e-850e-89fecc63ecfe-etc-sysctl-d\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.781479 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779494 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/475d8de2-f070-4c1e-850e-89fecc63ecfe-lib-modules\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.781479 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779517 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-etc-openvswitch\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.781479 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779540 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/27a3595e-6df7-4999-8257-5c111b074330-dbus\") pod \"global-pull-secret-syncer-bg9p7\" (UID: \"27a3595e-6df7-4999-8257-5c111b074330\") " pod="kube-system/global-pull-secret-syncer-bg9p7" Jul 02 21:36:05.781479 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779566 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnqz8\" (UniqueName: \"kubernetes.io/projected/298fd481-12c6-4869-a4d4-9b17c492706a-kube-api-access-cnqz8\") pod \"network-check-target-zkjd5\" (UID: \"298fd481-12c6-4869-a4d4-9b17c492706a\") " pod="openshift-network-diagnostics/network-check-target-zkjd5" Jul 02 21:36:05.781479 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779592 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-cnibin\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.781479 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779617 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6117c96e-d19e-48ab-a235-187d19f296d7-multus-daemon-config\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.782088 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779644 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eb1cecd6-d74b-4f21-bb0a-02f534221301-cnibin\") pod \"multus-additional-cni-plugins-qmwz4\" (UID: \"eb1cecd6-d74b-4f21-bb0a-02f534221301\") " pod="openshift-multus/multus-additional-cni-plugins-qmwz4" Jul 02 21:36:05.782088 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779705 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/eb1cecd6-d74b-4f21-bb0a-02f534221301-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qmwz4\" (UID: \"eb1cecd6-d74b-4f21-bb0a-02f534221301\") " pod="openshift-multus/multus-additional-cni-plugins-qmwz4" Jul 02 21:36:05.782088 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779731 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/475d8de2-f070-4c1e-850e-89fecc63ecfe-etc-modprobe-d\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.782088 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779755 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/475d8de2-f070-4c1e-850e-89fecc63ecfe-tmp\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.782088 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779778 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmbzg\" (UniqueName: \"kubernetes.io/projected/baf560bd-eaae-468e-bc1c-a007fe68bded-kube-api-access-cmbzg\") pod \"node-ca-82ctv\" (UID: \"baf560bd-eaae-468e-bc1c-a007fe68bded\") " pod="openshift-image-registry/node-ca-82ctv" Jul 02 21:36:05.782088 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779803 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-host-slash\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.782088 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779827 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-run-openvswitch\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.782088 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779851 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.782088 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779862 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7a32947e-1a4e-4138-8367-6def69748311-env-overrides\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.782088 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779897 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-multus-socket-dir-parent\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.782088 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.779976 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-etc-openvswitch\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.782088 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.780000 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eb1cecd6-d74b-4f21-bb0a-02f534221301-cnibin\") pod \"multus-additional-cni-plugins-qmwz4\" (UID: \"eb1cecd6-d74b-4f21-bb0a-02f534221301\") " pod="openshift-multus/multus-additional-cni-plugins-qmwz4" Jul 02 21:36:05.782088 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.780079 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/27a3595e-6df7-4999-8257-5c111b074330-dbus\") pod \"global-pull-secret-syncer-bg9p7\" (UID: \"27a3595e-6df7-4999-8257-5c111b074330\") " pod="kube-system/global-pull-secret-syncer-bg9p7" Jul 02 21:36:05.782088 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.780128 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-run-openvswitch\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.782088 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.780165 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/475d8de2-f070-4c1e-850e-89fecc63ecfe-etc-modprobe-d\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.782088 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.780214 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.782088 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.780272 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/475d8de2-f070-4c1e-850e-89fecc63ecfe-sys\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.782764 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.780305 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/baf560bd-eaae-468e-bc1c-a007fe68bded-host\") pod \"node-ca-82ctv\" (UID: \"baf560bd-eaae-468e-bc1c-a007fe68bded\") " pod="openshift-image-registry/node-ca-82ctv" Jul 02 21:36:05.782764 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.780451 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/eb1cecd6-d74b-4f21-bb0a-02f534221301-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qmwz4\" (UID: \"eb1cecd6-d74b-4f21-bb0a-02f534221301\") " pod="openshift-multus/multus-additional-cni-plugins-qmwz4" Jul 02 21:36:05.782764 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.780558 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7a32947e-1a4e-4138-8367-6def69748311-ovnkube-config\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.782764 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.780560 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/475d8de2-f070-4c1e-850e-89fecc63ecfe-etc-sysctl-d\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.782764 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.780589 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-host-slash\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.782764 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.780598 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/475d8de2-f070-4c1e-850e-89fecc63ecfe-lib-modules\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.782764 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.780627 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7a32947e-1a4e-4138-8367-6def69748311-host-cni-netd\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.782764 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.781160 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7a32947e-1a4e-4138-8367-6def69748311-ovnkube-script-lib\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.783191 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.782996 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/475d8de2-f070-4c1e-850e-89fecc63ecfe-tmp\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.783191 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.783080 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/475d8de2-f070-4c1e-850e-89fecc63ecfe-etc-tuned\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.784006 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.783984 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7a32947e-1a4e-4138-8367-6def69748311-ovn-node-metrics-cert\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.787027 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.785230 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/13dcc457-ac6d-416b-82e8-191fa49d0349-agent-certs\") pod \"konnectivity-agent-mtlf9\" (UID: \"13dcc457-ac6d-416b-82e8-191fa49d0349\") " pod="kube-system/konnectivity-agent-mtlf9" Jul 02 21:36:05.790965 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.790941 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7ts9\" (UniqueName: \"kubernetes.io/projected/475d8de2-f070-4c1e-850e-89fecc63ecfe-kube-api-access-g7ts9\") pod \"tuned-ntfv6\" (UID: \"475d8de2-f070-4c1e-850e-89fecc63ecfe\") " pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.791064 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.791044 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t52kg\" (UniqueName: \"kubernetes.io/projected/7a32947e-1a4e-4138-8367-6def69748311-kube-api-access-t52kg\") pod \"ovnkube-node-s8jx8\" (UID: \"7a32947e-1a4e-4138-8367-6def69748311\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:05.791237 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.791222 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t86k\" (UniqueName: \"kubernetes.io/projected/43a2d69d-8f93-4618-a04d-044c71f7e00b-kube-api-access-6t86k\") pod \"iptables-alerter-5hc6v\" (UID: \"43a2d69d-8f93-4618-a04d-044c71f7e00b\") " pod="openshift-network-operator/iptables-alerter-5hc6v" Jul 02 21:36:05.792112 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.792096 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmbzg\" (UniqueName: \"kubernetes.io/projected/baf560bd-eaae-468e-bc1c-a007fe68bded-kube-api-access-cmbzg\") pod \"node-ca-82ctv\" (UID: \"baf560bd-eaae-468e-bc1c-a007fe68bded\") " pod="openshift-image-registry/node-ca-82ctv" Jul 02 21:36:05.792187 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.792115 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h56r7\" (UniqueName: \"kubernetes.io/projected/eb1cecd6-d74b-4f21-bb0a-02f534221301-kube-api-access-h56r7\") pod \"multus-additional-cni-plugins-qmwz4\" (UID: \"eb1cecd6-d74b-4f21-bb0a-02f534221301\") " pod="openshift-multus/multus-additional-cni-plugins-qmwz4" Jul 02 21:36:05.793796 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.793774 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98czv\" (UniqueName: \"kubernetes.io/projected/cb5991b9-b1ea-46a5-9398-fe121b19bb4f-kube-api-access-98czv\") pod \"aws-ebs-csi-driver-node-f4vzj\" (UID: \"cb5991b9-b1ea-46a5-9398-fe121b19bb4f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f4vzj" Jul 02 21:36:05.815003 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.814960 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-146.ec2.internal" event={"ID":"8861d880e93a504d75b18448f12e97fd","Type":"ContainerStarted","Data":"c55840c87714aef610958575d504764cdef7d5dacdb819d9b2091e800358c3b3"} Jul 02 21:36:05.816201 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.816181 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-146.ec2.internal" event={"ID":"0670fa63672b3eeec21da85e068d962b","Type":"ContainerStarted","Data":"8180edf55a51ffad6d8209b527bf981dcb9c0e5ea569e30bf4100284226b99a7"} Jul 02 21:36:05.880568 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.880538 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6117c96e-d19e-48ab-a235-187d19f296d7-cni-binary-copy\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.880695 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.880574 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-host-var-lib-cni-bin\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.880695 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.880599 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtfh9\" (UniqueName: \"kubernetes.io/projected/6117c96e-d19e-48ab-a235-187d19f296d7-kube-api-access-gtfh9\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.880695 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.880624 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/222d3332-3472-479e-a8e2-8d8d9c702a9b-tmp-dir\") pod \"node-resolver-jgmz2\" (UID: \"222d3332-3472-479e-a8e2-8d8d9c702a9b\") " pod="openshift-dns/node-resolver-jgmz2" Jul 02 21:36:05.880695 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.880647 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-os-release\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.880695 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.880661 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-hostroot\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.880695 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.880672 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-host-var-lib-cni-bin\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.880695 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.880678 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnqz8\" (UniqueName: \"kubernetes.io/projected/298fd481-12c6-4869-a4d4-9b17c492706a-kube-api-access-cnqz8\") pod \"network-check-target-zkjd5\" (UID: \"298fd481-12c6-4869-a4d4-9b17c492706a\") " pod="openshift-network-diagnostics/network-check-target-zkjd5" Jul 02 21:36:05.881056 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.880716 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-cnibin\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.881056 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.880738 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6117c96e-d19e-48ab-a235-187d19f296d7-multus-daemon-config\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.881056 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.880769 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-multus-socket-dir-parent\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.881056 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.880804 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzhpf\" (UniqueName: \"kubernetes.io/projected/222d3332-3472-479e-a8e2-8d8d9c702a9b-kube-api-access-fzhpf\") pod \"node-resolver-jgmz2\" (UID: \"222d3332-3472-479e-a8e2-8d8d9c702a9b\") " pod="openshift-dns/node-resolver-jgmz2" Jul 02 21:36:05.881056 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.880856 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgwrx\" (UniqueName: \"kubernetes.io/projected/0bc3fefa-dff5-4181-a239-c6573b8b4356-kube-api-access-xgwrx\") pod \"network-metrics-daemon-zp6sf\" (UID: \"0bc3fefa-dff5-4181-a239-c6573b8b4356\") " pod="openshift-multus/network-metrics-daemon-zp6sf" Jul 02 21:36:05.881056 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.880902 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bc3fefa-dff5-4181-a239-c6573b8b4356-metrics-certs\") pod \"network-metrics-daemon-zp6sf\" (UID: \"0bc3fefa-dff5-4181-a239-c6573b8b4356\") " pod="openshift-multus/network-metrics-daemon-zp6sf" Jul 02 21:36:05.881056 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.880930 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-host-run-netns\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.881056 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.880948 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-multus-conf-dir\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.881056 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.880970 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-host-run-multus-certs\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.881056 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.880999 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-host-var-lib-kubelet\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.881056 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.881032 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-etc-kubernetes\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.881585 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.881061 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-system-cni-dir\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.881585 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.881105 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/222d3332-3472-479e-a8e2-8d8d9c702a9b-hosts-file\") pod \"node-resolver-jgmz2\" (UID: \"222d3332-3472-479e-a8e2-8d8d9c702a9b\") " pod="openshift-dns/node-resolver-jgmz2" Jul 02 21:36:05.881585 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.881124 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-host-var-lib-cni-multus\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.881585 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.881151 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-multus-cni-dir\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.881585 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.881159 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6117c96e-d19e-48ab-a235-187d19f296d7-cni-binary-copy\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.881585 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.881172 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-host-run-k8s-cni-cncf-io\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.881585 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.881208 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-host-run-k8s-cni-cncf-io\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.881585 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.881242 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-multus-conf-dir\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.881585 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.881241 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/222d3332-3472-479e-a8e2-8d8d9c702a9b-tmp-dir\") pod \"node-resolver-jgmz2\" (UID: \"222d3332-3472-479e-a8e2-8d8d9c702a9b\") " pod="openshift-dns/node-resolver-jgmz2" Jul 02 21:36:05.881585 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.881246 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-host-run-multus-certs\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.881585 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.881272 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-host-var-lib-kubelet\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.881585 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.881283 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-hostroot\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.881585 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.881303 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-cnibin\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.881585 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.881327 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-os-release\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.881585 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.881361 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-multus-socket-dir-parent\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.881585 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.881366 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-system-cni-dir\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.881585 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.881362 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-etc-kubernetes\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.881585 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.881412 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-host-var-lib-cni-multus\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.882484 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.881468 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-multus-cni-dir\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.882484 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:05.881528 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:36:05.882484 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.881557 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6117c96e-d19e-48ab-a235-187d19f296d7-host-run-netns\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.882484 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:05.881576 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bc3fefa-dff5-4181-a239-c6573b8b4356-metrics-certs podName:0bc3fefa-dff5-4181-a239-c6573b8b4356 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:06.381560689 +0000 UTC m=+3.100910209 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0bc3fefa-dff5-4181-a239-c6573b8b4356-metrics-certs") pod "network-metrics-daemon-zp6sf" (UID: "0bc3fefa-dff5-4181-a239-c6573b8b4356") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:36:05.882484 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.881603 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/222d3332-3472-479e-a8e2-8d8d9c702a9b-hosts-file\") pod \"node-resolver-jgmz2\" (UID: \"222d3332-3472-479e-a8e2-8d8d9c702a9b\") " pod="openshift-dns/node-resolver-jgmz2" Jul 02 21:36:05.882484 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.881744 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6117c96e-d19e-48ab-a235-187d19f296d7-multus-daemon-config\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.886594 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:05.886563 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 21:36:05.886594 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:05.886592 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 21:36:05.886734 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:05.886605 2556 projected.go:196] Error preparing data for projected volume kube-api-access-cnqz8 for pod openshift-network-diagnostics/network-check-target-zkjd5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:36:05.886734 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:05.886666 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/298fd481-12c6-4869-a4d4-9b17c492706a-kube-api-access-cnqz8 podName:298fd481-12c6-4869-a4d4-9b17c492706a nodeName:}" failed. No retries permitted until 2026-07-02 21:36:06.386654279 +0000 UTC m=+3.106003797 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-cnqz8" (UniqueName: "kubernetes.io/projected/298fd481-12c6-4869-a4d4-9b17c492706a-kube-api-access-cnqz8") pod "network-check-target-zkjd5" (UID: "298fd481-12c6-4869-a4d4-9b17c492706a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:36:05.888932 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.888909 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtfh9\" (UniqueName: \"kubernetes.io/projected/6117c96e-d19e-48ab-a235-187d19f296d7-kube-api-access-gtfh9\") pod \"multus-cdxjn\" (UID: \"6117c96e-d19e-48ab-a235-187d19f296d7\") " pod="openshift-multus/multus-cdxjn" Jul 02 21:36:05.889805 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.889777 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgwrx\" (UniqueName: \"kubernetes.io/projected/0bc3fefa-dff5-4181-a239-c6573b8b4356-kube-api-access-xgwrx\") pod \"network-metrics-daemon-zp6sf\" (UID: \"0bc3fefa-dff5-4181-a239-c6573b8b4356\") " pod="openshift-multus/network-metrics-daemon-zp6sf" Jul 02 21:36:05.889805 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.889802 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzhpf\" (UniqueName: \"kubernetes.io/projected/222d3332-3472-479e-a8e2-8d8d9c702a9b-kube-api-access-fzhpf\") pod \"node-resolver-jgmz2\" (UID: \"222d3332-3472-479e-a8e2-8d8d9c702a9b\") " pod="openshift-dns/node-resolver-jgmz2" Jul 02 21:36:05.966957 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.966891 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qmwz4" Jul 02 21:36:05.975009 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.974990 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5hc6v" Jul 02 21:36:05.985038 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.985019 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f4vzj" Jul 02 21:36:05.990573 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.990557 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" Jul 02 21:36:05.997374 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:05.997357 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-82ctv" Jul 02 21:36:06.005203 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:06.005186 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:06.014121 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:06.014104 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mtlf9" Jul 02 21:36:06.020917 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:06.020901 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cdxjn" Jul 02 21:36:06.025883 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:06.025856 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jgmz2" Jul 02 21:36:06.272503 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:06.272256 2556 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 21:36:06.284822 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:06.284792 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/27a3595e-6df7-4999-8257-5c111b074330-original-pull-secret\") pod \"global-pull-secret-syncer-bg9p7\" (UID: \"27a3595e-6df7-4999-8257-5c111b074330\") " pod="kube-system/global-pull-secret-syncer-bg9p7" Jul 02 21:36:06.284990 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:06.284940 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 21:36:06.285061 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:06.285003 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27a3595e-6df7-4999-8257-5c111b074330-original-pull-secret podName:27a3595e-6df7-4999-8257-5c111b074330 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:07.284985612 +0000 UTC m=+4.004335145 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/27a3595e-6df7-4999-8257-5c111b074330-original-pull-secret") pod "global-pull-secret-syncer-bg9p7" (UID: "27a3595e-6df7-4999-8257-5c111b074330") : object "kube-system"/"original-pull-secret" not registered Jul 02 21:36:06.385860 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:06.385824 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bc3fefa-dff5-4181-a239-c6573b8b4356-metrics-certs\") pod \"network-metrics-daemon-zp6sf\" (UID: \"0bc3fefa-dff5-4181-a239-c6573b8b4356\") " pod="openshift-multus/network-metrics-daemon-zp6sf" Jul 02 21:36:06.386010 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:06.385971 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:36:06.386058 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:06.386034 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bc3fefa-dff5-4181-a239-c6573b8b4356-metrics-certs podName:0bc3fefa-dff5-4181-a239-c6573b8b4356 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:07.386017204 +0000 UTC m=+4.105366723 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0bc3fefa-dff5-4181-a239-c6573b8b4356-metrics-certs") pod "network-metrics-daemon-zp6sf" (UID: "0bc3fefa-dff5-4181-a239-c6573b8b4356") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:36:06.486312 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:06.486282 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnqz8\" (UniqueName: \"kubernetes.io/projected/298fd481-12c6-4869-a4d4-9b17c492706a-kube-api-access-cnqz8\") pod \"network-check-target-zkjd5\" (UID: \"298fd481-12c6-4869-a4d4-9b17c492706a\") " pod="openshift-network-diagnostics/network-check-target-zkjd5" Jul 02 21:36:06.486511 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:06.486488 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 21:36:06.486560 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:06.486523 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 21:36:06.486560 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:06.486538 2556 projected.go:196] Error preparing data for projected volume kube-api-access-cnqz8 for pod openshift-network-diagnostics/network-check-target-zkjd5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:36:06.486663 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:06.486634 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/298fd481-12c6-4869-a4d4-9b17c492706a-kube-api-access-cnqz8 podName:298fd481-12c6-4869-a4d4-9b17c492706a nodeName:}" failed. No retries permitted until 2026-07-02 21:36:07.48661292 +0000 UTC m=+4.205962442 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-cnqz8" (UniqueName: "kubernetes.io/projected/298fd481-12c6-4869-a4d4-9b17c492706a-kube-api-access-cnqz8") pod "network-check-target-zkjd5" (UID: "298fd481-12c6-4869-a4d4-9b17c492706a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:36:06.720136 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:06.720064 2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 21:31:05 +0000 UTC" deadline="2028-01-30 14:43:36.46588227 +0000 UTC" Jul 02 21:36:06.720136 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:06.720092 2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13841h7m29.74579287s" Jul 02 21:36:06.746913 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:06.746865 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6117c96e_d19e_48ab_a235_187d19f296d7.slice/crio-69659b057c2a469854fe901b461edbed2e2a5810dbbf59898b4e3b3326cbdd04 WatchSource:0}: Error finding container 69659b057c2a469854fe901b461edbed2e2a5810dbbf59898b4e3b3326cbdd04: Status 404 returned error can't find the container with id 69659b057c2a469854fe901b461edbed2e2a5810dbbf59898b4e3b3326cbdd04 Jul 02 21:36:06.768308 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:06.768281 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43a2d69d_8f93_4618_a04d_044c71f7e00b.slice/crio-43434d0732ab0ffe7218d7bb0d001fdb50dc5bdf64af8e6aaa90a10de1efaf1b WatchSource:0}: Error finding container 43434d0732ab0ffe7218d7bb0d001fdb50dc5bdf64af8e6aaa90a10de1efaf1b: Status 404 returned error can't find the container with id 43434d0732ab0ffe7218d7bb0d001fdb50dc5bdf64af8e6aaa90a10de1efaf1b Jul 02 21:36:06.769833 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:06.769647 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod475d8de2_f070_4c1e_850e_89fecc63ecfe.slice/crio-7a869e0a82f4402d92276ca7c5e256bf9a526a8eb8656cda3bf92a0a6065fdf0 WatchSource:0}: Error finding container 7a869e0a82f4402d92276ca7c5e256bf9a526a8eb8656cda3bf92a0a6065fdf0: Status 404 returned error can't find the container with id 7a869e0a82f4402d92276ca7c5e256bf9a526a8eb8656cda3bf92a0a6065fdf0 Jul 02 21:36:06.770637 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:06.770410 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13dcc457_ac6d_416b_82e8_191fa49d0349.slice/crio-78b57836b71106e4f6743c86be6d5ca030a99fed5132f4c840eff176530b121f WatchSource:0}: Error finding container 78b57836b71106e4f6743c86be6d5ca030a99fed5132f4c840eff176530b121f: Status 404 returned error can't find the container with id 78b57836b71106e4f6743c86be6d5ca030a99fed5132f4c840eff176530b121f Jul 02 21:36:06.810681 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:06.810663 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zp6sf" Jul 02 21:36:06.810774 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:06.810758 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zp6sf" podUID="0bc3fefa-dff5-4181-a239-c6573b8b4356" Jul 02 21:36:06.818467 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:06.818443 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" event={"ID":"7a32947e-1a4e-4138-8367-6def69748311","Type":"ContainerStarted","Data":"0e909eeea79e2000b5ea7a185341ce8103a6cc9dbcdc0444923faae2a9c16555"} Jul 02 21:36:06.820183 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:06.820156 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cdxjn" event={"ID":"6117c96e-d19e-48ab-a235-187d19f296d7","Type":"ContainerStarted","Data":"69659b057c2a469854fe901b461edbed2e2a5810dbbf59898b4e3b3326cbdd04"} Jul 02 21:36:06.821538 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:06.821518 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qmwz4" event={"ID":"eb1cecd6-d74b-4f21-bb0a-02f534221301","Type":"ContainerStarted","Data":"4f16242a9394714b769c9c16e6a2af6db055e98ded4a871a5463e94f433530ed"} Jul 02 21:36:06.822953 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:06.822929 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f4vzj" event={"ID":"cb5991b9-b1ea-46a5-9398-fe121b19bb4f","Type":"ContainerStarted","Data":"f55120f04ae9aa1fc45ef69aaf3c9fa11df60010992f94902ef7fd8762de05ce"} Jul 02 21:36:06.823987 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:06.823967 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-82ctv" event={"ID":"baf560bd-eaae-468e-bc1c-a007fe68bded","Type":"ContainerStarted","Data":"359d8ed298d03639521bc97edd743f6c37650fdc986c29fa4dcaa7d439080459"} Jul 02 21:36:06.824914 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:06.824895 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jgmz2" event={"ID":"222d3332-3472-479e-a8e2-8d8d9c702a9b","Type":"ContainerStarted","Data":"075ce88e579e94e17655a4aa745ee57e8510913e1ef686ab6ec574dedf88658d"} Jul 02 21:36:06.825818 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:06.825798 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mtlf9" event={"ID":"13dcc457-ac6d-416b-82e8-191fa49d0349","Type":"ContainerStarted","Data":"78b57836b71106e4f6743c86be6d5ca030a99fed5132f4c840eff176530b121f"} Jul 02 21:36:06.826813 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:06.826794 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" event={"ID":"475d8de2-f070-4c1e-850e-89fecc63ecfe","Type":"ContainerStarted","Data":"7a869e0a82f4402d92276ca7c5e256bf9a526a8eb8656cda3bf92a0a6065fdf0"} Jul 02 21:36:06.830699 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:06.830677 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5hc6v" event={"ID":"43a2d69d-8f93-4618-a04d-044c71f7e00b","Type":"ContainerStarted","Data":"43434d0732ab0ffe7218d7bb0d001fdb50dc5bdf64af8e6aaa90a10de1efaf1b"} Jul 02 21:36:07.289333 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:07.289283 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/27a3595e-6df7-4999-8257-5c111b074330-original-pull-secret\") pod \"global-pull-secret-syncer-bg9p7\" (UID: \"27a3595e-6df7-4999-8257-5c111b074330\") " pod="kube-system/global-pull-secret-syncer-bg9p7" Jul 02 21:36:07.289513 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:07.289496 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 21:36:07.289574 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:07.289565 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27a3595e-6df7-4999-8257-5c111b074330-original-pull-secret podName:27a3595e-6df7-4999-8257-5c111b074330 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:09.28954879 +0000 UTC m=+6.008898310 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/27a3595e-6df7-4999-8257-5c111b074330-original-pull-secret") pod "global-pull-secret-syncer-bg9p7" (UID: "27a3595e-6df7-4999-8257-5c111b074330") : object "kube-system"/"original-pull-secret" not registered Jul 02 21:36:07.389800 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:07.389759 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bc3fefa-dff5-4181-a239-c6573b8b4356-metrics-certs\") pod \"network-metrics-daemon-zp6sf\" (UID: \"0bc3fefa-dff5-4181-a239-c6573b8b4356\") " pod="openshift-multus/network-metrics-daemon-zp6sf" Jul 02 21:36:07.390016 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:07.389977 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:36:07.390083 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:07.390038 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bc3fefa-dff5-4181-a239-c6573b8b4356-metrics-certs podName:0bc3fefa-dff5-4181-a239-c6573b8b4356 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:09.390019615 +0000 UTC m=+6.109369141 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0bc3fefa-dff5-4181-a239-c6573b8b4356-metrics-certs") pod "network-metrics-daemon-zp6sf" (UID: "0bc3fefa-dff5-4181-a239-c6573b8b4356") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:36:07.490540 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:07.490472 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnqz8\" (UniqueName: \"kubernetes.io/projected/298fd481-12c6-4869-a4d4-9b17c492706a-kube-api-access-cnqz8\") pod \"network-check-target-zkjd5\" (UID: \"298fd481-12c6-4869-a4d4-9b17c492706a\") " pod="openshift-network-diagnostics/network-check-target-zkjd5" Jul 02 21:36:07.490720 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:07.490655 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 21:36:07.490720 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:07.490679 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 21:36:07.490720 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:07.490691 2556 projected.go:196] Error preparing data for projected volume kube-api-access-cnqz8 for pod openshift-network-diagnostics/network-check-target-zkjd5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:36:07.490908 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:07.490753 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/298fd481-12c6-4869-a4d4-9b17c492706a-kube-api-access-cnqz8 podName:298fd481-12c6-4869-a4d4-9b17c492706a nodeName:}" failed. No retries permitted until 2026-07-02 21:36:09.490732283 +0000 UTC m=+6.210081807 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-cnqz8" (UniqueName: "kubernetes.io/projected/298fd481-12c6-4869-a4d4-9b17c492706a-kube-api-access-cnqz8") pod "network-check-target-zkjd5" (UID: "298fd481-12c6-4869-a4d4-9b17c492706a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:36:07.720751 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:07.720644 2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 21:31:05 +0000 UTC" deadline="2027-12-01 10:36:28.699017619 +0000 UTC" Jul 02 21:36:07.720751 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:07.720703 2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12397h0m20.978318995s" Jul 02 21:36:07.812057 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:07.812025 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkjd5" Jul 02 21:36:07.812245 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:07.812157 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkjd5" podUID="298fd481-12c6-4869-a4d4-9b17c492706a" Jul 02 21:36:07.812576 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:07.812533 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg9p7" Jul 02 21:36:07.812683 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:07.812626 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bg9p7" podUID="27a3595e-6df7-4999-8257-5c111b074330" Jul 02 21:36:07.883735 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:07.883606 2556 generic.go:356] "Generic (PLEG): container finished" podID="0670fa63672b3eeec21da85e068d962b" containerID="0162708867ba4ed46362927b4a113b8c4e4dad6911a2824619c672b803ba2466" exitCode=0 Jul 02 21:36:07.883735 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:07.883689 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-146.ec2.internal" event={"ID":"0670fa63672b3eeec21da85e068d962b","Type":"ContainerDied","Data":"0162708867ba4ed46362927b4a113b8c4e4dad6911a2824619c672b803ba2466"} Jul 02 21:36:07.900145 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:07.899945 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-146.ec2.internal" event={"ID":"8861d880e93a504d75b18448f12e97fd","Type":"ContainerStarted","Data":"7b490eb10ecbeaba7185251891970519d435efa1cfbfdab50699195e2a49514d"} Jul 02 21:36:07.939864 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:07.939466 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-146.ec2.internal" podStartSLOduration=3.939447515 podStartE2EDuration="3.939447515s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 21:36:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:36:07.939347671 +0000 UTC m=+4.658697219" watchObservedRunningTime="2026-07-02 21:36:07.939447515 +0000 UTC m=+4.658797057" Jul 02 21:36:08.810503 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:08.810036 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zp6sf" Jul 02 21:36:08.810503 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:08.810171 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zp6sf" podUID="0bc3fefa-dff5-4181-a239-c6573b8b4356" Jul 02 21:36:08.913068 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:08.912991 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-146.ec2.internal" event={"ID":"0670fa63672b3eeec21da85e068d962b","Type":"ContainerStarted","Data":"13670b4ee957cbc09787083261ed8aae2e1d75069f9cfaf38895f4b6ce588e4b"} Jul 02 21:36:09.306631 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:09.306051 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/27a3595e-6df7-4999-8257-5c111b074330-original-pull-secret\") pod \"global-pull-secret-syncer-bg9p7\" (UID: \"27a3595e-6df7-4999-8257-5c111b074330\") " pod="kube-system/global-pull-secret-syncer-bg9p7" Jul 02 21:36:09.306631 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:09.306191 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 21:36:09.306631 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:09.306252 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27a3595e-6df7-4999-8257-5c111b074330-original-pull-secret podName:27a3595e-6df7-4999-8257-5c111b074330 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:13.306232944 +0000 UTC m=+10.025582485 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/27a3595e-6df7-4999-8257-5c111b074330-original-pull-secret") pod "global-pull-secret-syncer-bg9p7" (UID: "27a3595e-6df7-4999-8257-5c111b074330") : object "kube-system"/"original-pull-secret" not registered Jul 02 21:36:09.407580 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:09.406888 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bc3fefa-dff5-4181-a239-c6573b8b4356-metrics-certs\") pod \"network-metrics-daemon-zp6sf\" (UID: \"0bc3fefa-dff5-4181-a239-c6573b8b4356\") " pod="openshift-multus/network-metrics-daemon-zp6sf" Jul 02 21:36:09.407580 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:09.407125 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:36:09.407580 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:09.407191 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bc3fefa-dff5-4181-a239-c6573b8b4356-metrics-certs podName:0bc3fefa-dff5-4181-a239-c6573b8b4356 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:13.407172574 +0000 UTC m=+10.126522110 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0bc3fefa-dff5-4181-a239-c6573b8b4356-metrics-certs") pod "network-metrics-daemon-zp6sf" (UID: "0bc3fefa-dff5-4181-a239-c6573b8b4356") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:36:09.507551 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:09.507507 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnqz8\" (UniqueName: \"kubernetes.io/projected/298fd481-12c6-4869-a4d4-9b17c492706a-kube-api-access-cnqz8\") pod \"network-check-target-zkjd5\" (UID: \"298fd481-12c6-4869-a4d4-9b17c492706a\") " pod="openshift-network-diagnostics/network-check-target-zkjd5" Jul 02 21:36:09.507743 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:09.507723 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 21:36:09.507830 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:09.507751 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 21:36:09.507830 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:09.507764 2556 projected.go:196] Error preparing data for projected volume kube-api-access-cnqz8 for pod openshift-network-diagnostics/network-check-target-zkjd5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:36:09.507830 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:09.507820 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/298fd481-12c6-4869-a4d4-9b17c492706a-kube-api-access-cnqz8 podName:298fd481-12c6-4869-a4d4-9b17c492706a nodeName:}" failed. No retries permitted until 2026-07-02 21:36:13.507802155 +0000 UTC m=+10.227151694 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-cnqz8" (UniqueName: "kubernetes.io/projected/298fd481-12c6-4869-a4d4-9b17c492706a-kube-api-access-cnqz8") pod "network-check-target-zkjd5" (UID: "298fd481-12c6-4869-a4d4-9b17c492706a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:36:09.812091 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:09.812058 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkjd5" Jul 02 21:36:09.812520 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:09.812270 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkjd5" podUID="298fd481-12c6-4869-a4d4-9b17c492706a" Jul 02 21:36:09.812520 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:09.812381 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg9p7" Jul 02 21:36:09.812620 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:09.812520 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bg9p7" podUID="27a3595e-6df7-4999-8257-5c111b074330" Jul 02 21:36:10.811265 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:10.810778 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zp6sf" Jul 02 21:36:10.811265 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:10.810936 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zp6sf" podUID="0bc3fefa-dff5-4181-a239-c6573b8b4356" Jul 02 21:36:11.815550 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:11.814998 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg9p7" Jul 02 21:36:11.815550 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:11.815103 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bg9p7" podUID="27a3595e-6df7-4999-8257-5c111b074330" Jul 02 21:36:11.815550 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:11.815410 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkjd5" Jul 02 21:36:11.815550 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:11.815473 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkjd5" podUID="298fd481-12c6-4869-a4d4-9b17c492706a" Jul 02 21:36:12.811286 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:12.810801 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zp6sf" Jul 02 21:36:12.811286 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:12.810949 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zp6sf" podUID="0bc3fefa-dff5-4181-a239-c6573b8b4356" Jul 02 21:36:13.340237 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:13.340201 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/27a3595e-6df7-4999-8257-5c111b074330-original-pull-secret\") pod \"global-pull-secret-syncer-bg9p7\" (UID: \"27a3595e-6df7-4999-8257-5c111b074330\") " pod="kube-system/global-pull-secret-syncer-bg9p7" Jul 02 21:36:13.340603 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:13.340381 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 21:36:13.340603 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:13.340454 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27a3595e-6df7-4999-8257-5c111b074330-original-pull-secret podName:27a3595e-6df7-4999-8257-5c111b074330 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:21.340433328 +0000 UTC m=+18.059782853 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/27a3595e-6df7-4999-8257-5c111b074330-original-pull-secret") pod "global-pull-secret-syncer-bg9p7" (UID: "27a3595e-6df7-4999-8257-5c111b074330") : object "kube-system"/"original-pull-secret" not registered Jul 02 21:36:13.441761 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:13.441069 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bc3fefa-dff5-4181-a239-c6573b8b4356-metrics-certs\") pod \"network-metrics-daemon-zp6sf\" (UID: \"0bc3fefa-dff5-4181-a239-c6573b8b4356\") " pod="openshift-multus/network-metrics-daemon-zp6sf" Jul 02 21:36:13.441761 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:13.441267 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:36:13.441761 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:13.441329 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bc3fefa-dff5-4181-a239-c6573b8b4356-metrics-certs podName:0bc3fefa-dff5-4181-a239-c6573b8b4356 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:21.441310991 +0000 UTC m=+18.160660512 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0bc3fefa-dff5-4181-a239-c6573b8b4356-metrics-certs") pod "network-metrics-daemon-zp6sf" (UID: "0bc3fefa-dff5-4181-a239-c6573b8b4356") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:36:13.541656 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:13.541621 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnqz8\" (UniqueName: \"kubernetes.io/projected/298fd481-12c6-4869-a4d4-9b17c492706a-kube-api-access-cnqz8\") pod \"network-check-target-zkjd5\" (UID: \"298fd481-12c6-4869-a4d4-9b17c492706a\") " pod="openshift-network-diagnostics/network-check-target-zkjd5" Jul 02 21:36:13.541858 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:13.541796 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 21:36:13.541858 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:13.541817 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 21:36:13.541858 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:13.541826 2556 projected.go:196] Error preparing data for projected volume kube-api-access-cnqz8 for pod openshift-network-diagnostics/network-check-target-zkjd5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:36:13.542038 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:13.541884 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/298fd481-12c6-4869-a4d4-9b17c492706a-kube-api-access-cnqz8 podName:298fd481-12c6-4869-a4d4-9b17c492706a nodeName:}" failed. No retries permitted until 2026-07-02 21:36:21.541857433 +0000 UTC m=+18.261206952 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-cnqz8" (UniqueName: "kubernetes.io/projected/298fd481-12c6-4869-a4d4-9b17c492706a-kube-api-access-cnqz8") pod "network-check-target-zkjd5" (UID: "298fd481-12c6-4869-a4d4-9b17c492706a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:36:13.813127 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:13.813100 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkjd5" Jul 02 21:36:13.813285 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:13.813185 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkjd5" podUID="298fd481-12c6-4869-a4d4-9b17c492706a" Jul 02 21:36:13.813471 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:13.813437 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg9p7" Jul 02 21:36:13.813530 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:13.813510 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bg9p7" podUID="27a3595e-6df7-4999-8257-5c111b074330" Jul 02 21:36:14.810287 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:14.810250 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zp6sf" Jul 02 21:36:14.810694 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:14.810396 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zp6sf" podUID="0bc3fefa-dff5-4181-a239-c6573b8b4356" Jul 02 21:36:15.810162 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:15.810127 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg9p7" Jul 02 21:36:15.810331 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:15.810265 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bg9p7" podUID="27a3595e-6df7-4999-8257-5c111b074330" Jul 02 21:36:15.810737 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:15.810704 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkjd5" Jul 02 21:36:15.810826 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:15.810793 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkjd5" podUID="298fd481-12c6-4869-a4d4-9b17c492706a" Jul 02 21:36:16.810927 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:16.810890 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zp6sf" Jul 02 21:36:16.811371 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:16.811030 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zp6sf" podUID="0bc3fefa-dff5-4181-a239-c6573b8b4356" Jul 02 21:36:17.810837 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:17.810795 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkjd5" Jul 02 21:36:17.811031 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:17.810930 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkjd5" podUID="298fd481-12c6-4869-a4d4-9b17c492706a" Jul 02 21:36:17.811031 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:17.810973 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg9p7" Jul 02 21:36:17.811031 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:17.811025 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bg9p7" podUID="27a3595e-6df7-4999-8257-5c111b074330" Jul 02 21:36:18.811014 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:18.810985 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zp6sf" Jul 02 21:36:18.811170 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:18.811096 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zp6sf" podUID="0bc3fefa-dff5-4181-a239-c6573b8b4356" Jul 02 21:36:19.810846 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:19.810810 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg9p7" Jul 02 21:36:19.811016 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:19.810810 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkjd5" Jul 02 21:36:19.811016 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:19.810969 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bg9p7" podUID="27a3595e-6df7-4999-8257-5c111b074330" Jul 02 21:36:19.811110 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:19.811022 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkjd5" podUID="298fd481-12c6-4869-a4d4-9b17c492706a" Jul 02 21:36:20.811042 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:20.811013 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zp6sf" Jul 02 21:36:20.811464 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:20.811127 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zp6sf" podUID="0bc3fefa-dff5-4181-a239-c6573b8b4356" Jul 02 21:36:21.398704 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:21.398672 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/27a3595e-6df7-4999-8257-5c111b074330-original-pull-secret\") pod \"global-pull-secret-syncer-bg9p7\" (UID: \"27a3595e-6df7-4999-8257-5c111b074330\") " pod="kube-system/global-pull-secret-syncer-bg9p7" Jul 02 21:36:21.398974 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:21.398803 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 21:36:21.398974 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:21.398892 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27a3595e-6df7-4999-8257-5c111b074330-original-pull-secret podName:27a3595e-6df7-4999-8257-5c111b074330 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:37.398852269 +0000 UTC m=+34.118201793 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/27a3595e-6df7-4999-8257-5c111b074330-original-pull-secret") pod "global-pull-secret-syncer-bg9p7" (UID: "27a3595e-6df7-4999-8257-5c111b074330") : object "kube-system"/"original-pull-secret" not registered Jul 02 21:36:21.499056 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:21.499016 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bc3fefa-dff5-4181-a239-c6573b8b4356-metrics-certs\") pod \"network-metrics-daemon-zp6sf\" (UID: \"0bc3fefa-dff5-4181-a239-c6573b8b4356\") " pod="openshift-multus/network-metrics-daemon-zp6sf" Jul 02 21:36:21.499244 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:21.499151 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:36:21.499244 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:21.499217 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bc3fefa-dff5-4181-a239-c6573b8b4356-metrics-certs podName:0bc3fefa-dff5-4181-a239-c6573b8b4356 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:37.499197212 +0000 UTC m=+34.218546738 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0bc3fefa-dff5-4181-a239-c6573b8b4356-metrics-certs") pod "network-metrics-daemon-zp6sf" (UID: "0bc3fefa-dff5-4181-a239-c6573b8b4356") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:36:21.599621 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:21.599582 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnqz8\" (UniqueName: \"kubernetes.io/projected/298fd481-12c6-4869-a4d4-9b17c492706a-kube-api-access-cnqz8\") pod \"network-check-target-zkjd5\" (UID: \"298fd481-12c6-4869-a4d4-9b17c492706a\") " pod="openshift-network-diagnostics/network-check-target-zkjd5" Jul 02 21:36:21.599793 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:21.599773 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 21:36:21.599839 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:21.599804 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 21:36:21.599839 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:21.599818 2556 projected.go:196] Error preparing data for projected volume kube-api-access-cnqz8 for pod openshift-network-diagnostics/network-check-target-zkjd5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:36:21.599993 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:21.599897 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/298fd481-12c6-4869-a4d4-9b17c492706a-kube-api-access-cnqz8 podName:298fd481-12c6-4869-a4d4-9b17c492706a nodeName:}" failed. No retries permitted until 2026-07-02 21:36:37.599863684 +0000 UTC m=+34.319213222 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-cnqz8" (UniqueName: "kubernetes.io/projected/298fd481-12c6-4869-a4d4-9b17c492706a-kube-api-access-cnqz8") pod "network-check-target-zkjd5" (UID: "298fd481-12c6-4869-a4d4-9b17c492706a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:36:21.810656 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:21.810623 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg9p7" Jul 02 21:36:21.810827 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:21.810737 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bg9p7" podUID="27a3595e-6df7-4999-8257-5c111b074330" Jul 02 21:36:21.810917 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:21.810825 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkjd5" Jul 02 21:36:21.810977 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:21.810949 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkjd5" podUID="298fd481-12c6-4869-a4d4-9b17c492706a" Jul 02 21:36:22.810257 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:22.810218 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zp6sf" Jul 02 21:36:22.810705 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:22.810417 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zp6sf" podUID="0bc3fefa-dff5-4181-a239-c6573b8b4356" Jul 02 21:36:23.810835 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:23.810799 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg9p7" Jul 02 21:36:23.811257 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:23.810905 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkjd5" Jul 02 21:36:23.811257 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:23.810913 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bg9p7" podUID="27a3595e-6df7-4999-8257-5c111b074330" Jul 02 21:36:23.811257 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:23.811019 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkjd5" podUID="298fd481-12c6-4869-a4d4-9b17c492706a" Jul 02 21:36:24.810430 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:24.810393 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zp6sf" Jul 02 21:36:24.810593 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:24.810524 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zp6sf" podUID="0bc3fefa-dff5-4181-a239-c6573b8b4356" Jul 02 21:36:25.810509 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:25.810476 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkjd5" Jul 02 21:36:25.810977 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:25.810476 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg9p7" Jul 02 21:36:25.810977 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:25.810576 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkjd5" podUID="298fd481-12c6-4869-a4d4-9b17c492706a" Jul 02 21:36:25.810977 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:25.810678 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bg9p7" podUID="27a3595e-6df7-4999-8257-5c111b074330" Jul 02 21:36:26.810965 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:26.810656 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zp6sf" Jul 02 21:36:26.811680 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:26.811061 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zp6sf" podUID="0bc3fefa-dff5-4181-a239-c6573b8b4356" Jul 02 21:36:26.947292 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:26.947207 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mtlf9" event={"ID":"13dcc457-ac6d-416b-82e8-191fa49d0349","Type":"ContainerStarted","Data":"e8245b3bbe99a58519d0a6e0c775768d9249a890f2c413cc82e927d76367dd5b"} Jul 02 21:36:26.949216 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:26.948907 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" event={"ID":"475d8de2-f070-4c1e-850e-89fecc63ecfe","Type":"ContainerStarted","Data":"bcb6fa660d436bf1e2b3b90ffa321e7ed1412155a7d932073fd9eee4924898ea"} Jul 02 21:36:26.952512 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:26.952491 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8jx8_7a32947e-1a4e-4138-8367-6def69748311/ovn-acl-logging/0.log" Jul 02 21:36:26.952890 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:26.952851 2556 generic.go:356] "Generic (PLEG): container finished" podID="7a32947e-1a4e-4138-8367-6def69748311" containerID="6d5d851f4f7dea91e36ce25b11e7c7a532c044ac2b6171407869c19ed2b3dd0f" exitCode=1 Jul 02 21:36:26.952980 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:26.952895 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" event={"ID":"7a32947e-1a4e-4138-8367-6def69748311","Type":"ContainerStarted","Data":"bf9034bbdef431a94dc12fc7aef7ca30112ae0d96f9e4c6aecbc0e56f0f396ba"} Jul 02 21:36:26.952980 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:26.952975 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" event={"ID":"7a32947e-1a4e-4138-8367-6def69748311","Type":"ContainerStarted","Data":"dcff735847ae64803a2a3bf127924e0f81756a5027bd46437038ba1f3ea7e6b7"} Jul 02 21:36:26.953077 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:26.953018 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" event={"ID":"7a32947e-1a4e-4138-8367-6def69748311","Type":"ContainerStarted","Data":"4df04dc5b6392e45b0905d1c3efa3be279dabe9147917406fd66cbc942053bcd"} Jul 02 21:36:26.953077 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:26.953066 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" event={"ID":"7a32947e-1a4e-4138-8367-6def69748311","Type":"ContainerStarted","Data":"441730a7b63fe34b8168c97131cd7251c10b3f6744604e32f3bab746f8fef624"} Jul 02 21:36:26.953169 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:26.953096 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" event={"ID":"7a32947e-1a4e-4138-8367-6def69748311","Type":"ContainerDied","Data":"6d5d851f4f7dea91e36ce25b11e7c7a532c044ac2b6171407869c19ed2b3dd0f"} Jul 02 21:36:26.953169 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:26.953131 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" event={"ID":"7a32947e-1a4e-4138-8367-6def69748311","Type":"ContainerStarted","Data":"47d83ad14a0ee2a57fc067b8463e48f33ba6e2610d84a0e3d86407b280082081"} Jul 02 21:36:26.954501 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:26.954482 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cdxjn" event={"ID":"6117c96e-d19e-48ab-a235-187d19f296d7","Type":"ContainerStarted","Data":"d98d05bd03a85d1ca1c407115ed8aaa777934fe520add0bbd875403f462a5e80"} Jul 02 21:36:26.956067 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:26.956044 2556 generic.go:356] "Generic (PLEG): container finished" podID="eb1cecd6-d74b-4f21-bb0a-02f534221301" containerID="4539b345c824979753ab913347e2fdb50b221f71bc148a6b2f72c44768d9c391" exitCode=0 Jul 02 21:36:26.956158 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:26.956131 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qmwz4" event={"ID":"eb1cecd6-d74b-4f21-bb0a-02f534221301","Type":"ContainerDied","Data":"4539b345c824979753ab913347e2fdb50b221f71bc148a6b2f72c44768d9c391"} Jul 02 21:36:26.957781 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:26.957757 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f4vzj" event={"ID":"cb5991b9-b1ea-46a5-9398-fe121b19bb4f","Type":"ContainerStarted","Data":"62186a9d10ba1dc48f843eeec0aeb0b453600ed96b1a00dfe0e9bdd2a95810e7"} Jul 02 21:36:26.959577 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:26.959541 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-82ctv" event={"ID":"baf560bd-eaae-468e-bc1c-a007fe68bded","Type":"ContainerStarted","Data":"6c42b154a46afe9e17f321e3eadf0e97a974f749289f2affb5bf96241d447926"} Jul 02 21:36:26.961154 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:26.961134 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jgmz2" event={"ID":"222d3332-3472-479e-a8e2-8d8d9c702a9b","Type":"ContainerStarted","Data":"3acb672944e64c0ae7acbeb13b1c155399fd44353a5a54fb4d2d774f4b4c6da0"} Jul 02 21:36:26.962841 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:26.962794 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-mtlf9" podStartSLOduration=13.280307192 podStartE2EDuration="23.962777125s" totalImagesPullingTime="10.682469933s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:36:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:36:26.961989139 +0000 UTC m=+23.681338681" watchObservedRunningTime="2026-07-02 21:36:26.962777125 +0000 UTC m=+23.682126667" Jul 02 21:36:26.963278 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:26.963245 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-146.ec2.internal" podStartSLOduration=22.963235993 podStartE2EDuration="22.963235993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 21:36:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:36:08.929631459 +0000 UTC m=+5.648981095" watchObservedRunningTime="2026-07-02 21:36:26.963235993 +0000 UTC m=+23.682585535" Jul 02 21:36:26.999325 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:26.999286 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-cdxjn" podStartSLOduration=4.743898147 podStartE2EDuration="23.999270632s" totalImagesPullingTime="19.255372485s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 21:36:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:36:26.998920936 +0000 UTC m=+23.718270478" watchObservedRunningTime="2026-07-02 21:36:26.999270632 +0000 UTC m=+23.718620173" Jul 02 21:36:27.033279 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:27.033230 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ntfv6" podStartSLOduration=4.805968502 podStartE2EDuration="24.033217354s" totalImagesPullingTime="19.227248852s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 21:36:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:36:27.019337244 +0000 UTC m=+23.738686784" watchObservedRunningTime="2026-07-02 21:36:27.033217354 +0000 UTC m=+23.752566936" Jul 02 21:36:27.033413 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:27.033339 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-82ctv" podStartSLOduration=4.814172743 podStartE2EDuration="24.03333459s" totalImagesPullingTime="19.219161847s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 21:36:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:36:27.032909459 +0000 UTC m=+23.752259073" watchObservedRunningTime="2026-07-02 21:36:27.03333459 +0000 UTC m=+23.752684130" Jul 02 21:36:27.046949 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:27.046911 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-jgmz2" podStartSLOduration=4.841946718 podStartE2EDuration="24.046898631s" totalImagesPullingTime="19.204951913s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 21:36:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:36:27.046673936 +0000 UTC m=+23.766023477" watchObservedRunningTime="2026-07-02 21:36:27.046898631 +0000 UTC m=+23.766248171" Jul 02 21:36:27.545048 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:27.544841 2556 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 02 21:36:27.740168 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:27.740009 2556 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-02T21:36:27.545043929Z","UUID":"f7256954-fe31-4d37-b6f1-47d43763c0b8","Handler":null,"Name":"","Endpoint":""} Jul 02 21:36:27.742003 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:27.741984 2556 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 02 21:36:27.742139 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:27.742023 2556 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 02 21:36:27.810904 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:27.810519 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg9p7" Jul 02 21:36:27.810904 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:27.810659 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bg9p7" podUID="27a3595e-6df7-4999-8257-5c111b074330" Jul 02 21:36:27.810904 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:27.810713 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkjd5" Jul 02 21:36:27.810904 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:27.810816 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkjd5" podUID="298fd481-12c6-4869-a4d4-9b17c492706a" Jul 02 21:36:27.965949 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:27.965907 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f4vzj" event={"ID":"cb5991b9-b1ea-46a5-9398-fe121b19bb4f","Type":"ContainerStarted","Data":"7e1e21ba47ab95159f0c6c59b4103b735a63c0b3c904834f99a0e1f656b23d41"} Jul 02 21:36:27.968235 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:27.968194 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5hc6v" event={"ID":"43a2d69d-8f93-4618-a04d-044c71f7e00b","Type":"ContainerStarted","Data":"17f6cb73e0449d5b4226629a35faffaedfeee4c4c72cd7c9caac5b2d450954d9"} Jul 02 21:36:27.982454 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:27.982402 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-5hc6v" podStartSLOduration=5.773848888 podStartE2EDuration="24.982384193s" totalImagesPullingTime="19.208535305s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 21:36:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:36:27.982107686 +0000 UTC m=+24.701457230" watchObservedRunningTime="2026-07-02 21:36:27.982384193 +0000 UTC m=+24.701733736" Jul 02 21:36:28.810236 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:28.810207 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-mtlf9" Jul 02 21:36:28.810424 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:28.810340 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zp6sf" Jul 02 21:36:28.810499 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:28.810476 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zp6sf" podUID="0bc3fefa-dff5-4181-a239-c6573b8b4356" Jul 02 21:36:28.810912 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:28.810884 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-mtlf9" Jul 02 21:36:28.971514 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:28.971487 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f4vzj" event={"ID":"cb5991b9-b1ea-46a5-9398-fe121b19bb4f","Type":"ContainerStarted","Data":"ff5ab3c1efb13a3db9131c5010d06360eae1a5930b8da623e68b734f4fe97d65"} Jul 02 21:36:28.972157 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:28.972139 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-mtlf9" Jul 02 21:36:28.972569 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:28.972556 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-mtlf9" Jul 02 21:36:28.989258 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:28.989213 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f4vzj" podStartSLOduration=4.568551321 podStartE2EDuration="25.989200971s" totalImagesPullingTime="21.42064965s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 21:36:03 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:36:28.988805114 +0000 UTC m=+25.708154655" watchObservedRunningTime="2026-07-02 21:36:28.989200971 +0000 UTC m=+25.708550540" Jul 02 21:36:29.810404 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:29.810365 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkjd5" Jul 02 21:36:29.810404 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:29.810365 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg9p7" Jul 02 21:36:29.810758 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:29.810495 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkjd5" podUID="298fd481-12c6-4869-a4d4-9b17c492706a" Jul 02 21:36:29.810758 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:29.810649 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bg9p7" podUID="27a3595e-6df7-4999-8257-5c111b074330" Jul 02 21:36:29.977435 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:29.977409 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8jx8_7a32947e-1a4e-4138-8367-6def69748311/ovn-acl-logging/0.log" Jul 02 21:36:29.978406 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:29.978375 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" event={"ID":"7a32947e-1a4e-4138-8367-6def69748311","Type":"ContainerStarted","Data":"f5a06de8fbc5e279756d802c191bb2fc609352367db544c33db372baff44ee54"} Jul 02 21:36:30.810461 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:30.810428 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zp6sf" Jul 02 21:36:30.810628 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:30.810548 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zp6sf" podUID="0bc3fefa-dff5-4181-a239-c6573b8b4356" Jul 02 21:36:31.810267 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:31.810044 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg9p7" Jul 02 21:36:31.810825 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:31.810100 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkjd5" Jul 02 21:36:31.810825 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:31.810378 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bg9p7" podUID="27a3595e-6df7-4999-8257-5c111b074330" Jul 02 21:36:31.810825 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:31.810404 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkjd5" podUID="298fd481-12c6-4869-a4d4-9b17c492706a" Jul 02 21:36:31.985061 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:31.985035 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8jx8_7a32947e-1a4e-4138-8367-6def69748311/ovn-acl-logging/0.log" Jul 02 21:36:31.985436 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:31.985407 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" event={"ID":"7a32947e-1a4e-4138-8367-6def69748311","Type":"ContainerStarted","Data":"aeff7b5a3a0f0528a21e0a9478a2fca09e7c883fb3139792b4f1e5e6f1beb464"} Jul 02 21:36:31.985672 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:31.985641 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:31.985753 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:31.985717 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:31.985901 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:31.985863 2556 scope.go:117] "RemoveContainer" containerID="6d5d851f4f7dea91e36ce25b11e7c7a532c044ac2b6171407869c19ed2b3dd0f" Jul 02 21:36:31.987449 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:31.987425 2556 generic.go:356] "Generic (PLEG): container finished" podID="eb1cecd6-d74b-4f21-bb0a-02f534221301" containerID="02ea813e731ffcf497c521fb75663cd54596b3b37ad6f0b063f0551c31f01591" exitCode=0 Jul 02 21:36:31.987540 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:31.987464 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qmwz4" event={"ID":"eb1cecd6-d74b-4f21-bb0a-02f534221301","Type":"ContainerDied","Data":"02ea813e731ffcf497c521fb75663cd54596b3b37ad6f0b063f0551c31f01591"} Jul 02 21:36:32.002070 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:32.002047 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:32.002218 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:32.002204 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:32.810095 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:32.810070 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zp6sf" Jul 02 21:36:32.810226 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:32.810169 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zp6sf" podUID="0bc3fefa-dff5-4181-a239-c6573b8b4356" Jul 02 21:36:32.992196 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:32.992163 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qmwz4" event={"ID":"eb1cecd6-d74b-4f21-bb0a-02f534221301","Type":"ContainerStarted","Data":"85ffa8d35f49634c1b4a60609ef4a353f35db8d35c6c287de130ac706f25bda6"} Jul 02 21:36:32.996733 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:32.996716 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8jx8_7a32947e-1a4e-4138-8367-6def69748311/ovn-acl-logging/0.log" Jul 02 21:36:32.997095 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:32.997076 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" event={"ID":"7a32947e-1a4e-4138-8367-6def69748311","Type":"ContainerStarted","Data":"7e39f78e8ebb4d9445b138dd0898899fdd08284a5ab56d797fafbaa9d0c0d204"} Jul 02 21:36:32.997423 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:32.997405 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:36:33.027175 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:33.026626 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" podStartSLOduration=10.56640644 podStartE2EDuration="30.026608018s" totalImagesPullingTime="19.460201578s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 21:36:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:36:33.025799536 +0000 UTC m=+29.745149078" watchObservedRunningTime="2026-07-02 21:36:33.026608018 +0000 UTC m=+29.745957560" Jul 02 21:36:33.201994 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:33.201926 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bg9p7"] Jul 02 21:36:33.202165 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:33.202050 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg9p7" Jul 02 21:36:33.202165 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:33.202133 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bg9p7" podUID="27a3595e-6df7-4999-8257-5c111b074330" Jul 02 21:36:33.205005 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:33.204978 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zkjd5"] Jul 02 21:36:33.205129 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:33.205078 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkjd5" Jul 02 21:36:33.205190 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:33.205170 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkjd5" podUID="298fd481-12c6-4869-a4d4-9b17c492706a" Jul 02 21:36:33.205579 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:33.205562 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zp6sf"] Jul 02 21:36:33.205653 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:33.205641 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zp6sf" Jul 02 21:36:33.205734 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:33.205720 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zp6sf" podUID="0bc3fefa-dff5-4181-a239-c6573b8b4356" Jul 02 21:36:34.000827 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:34.000795 2556 generic.go:356] "Generic (PLEG): container finished" podID="eb1cecd6-d74b-4f21-bb0a-02f534221301" containerID="85ffa8d35f49634c1b4a60609ef4a353f35db8d35c6c287de130ac706f25bda6" exitCode=0 Jul 02 21:36:34.001392 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:34.000888 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qmwz4" event={"ID":"eb1cecd6-d74b-4f21-bb0a-02f534221301","Type":"ContainerDied","Data":"85ffa8d35f49634c1b4a60609ef4a353f35db8d35c6c287de130ac706f25bda6"} Jul 02 21:36:34.810351 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:34.810318 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkjd5" Jul 02 21:36:34.810514 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:34.810407 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg9p7" Jul 02 21:36:34.810514 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:34.810418 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zp6sf" Jul 02 21:36:34.810514 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:34.810413 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkjd5" podUID="298fd481-12c6-4869-a4d4-9b17c492706a" Jul 02 21:36:34.810514 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:34.810484 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bg9p7" podUID="27a3595e-6df7-4999-8257-5c111b074330" Jul 02 21:36:34.810657 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:34.810561 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zp6sf" podUID="0bc3fefa-dff5-4181-a239-c6573b8b4356" Jul 02 21:36:36.007526 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:36.007497 2556 generic.go:356] "Generic (PLEG): container finished" podID="eb1cecd6-d74b-4f21-bb0a-02f534221301" containerID="059eea9cf0cc490ac71b67ba8d103645ea5b7f57f0aac8c050f4d45ccdc5d0d1" exitCode=0 Jul 02 21:36:36.008135 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:36.007553 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qmwz4" event={"ID":"eb1cecd6-d74b-4f21-bb0a-02f534221301","Type":"ContainerDied","Data":"059eea9cf0cc490ac71b67ba8d103645ea5b7f57f0aac8c050f4d45ccdc5d0d1"} Jul 02 21:36:36.810082 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:36.810050 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkjd5" Jul 02 21:36:36.810247 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:36.810050 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg9p7" Jul 02 21:36:36.810247 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:36.810188 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkjd5" podUID="298fd481-12c6-4869-a4d4-9b17c492706a" Jul 02 21:36:36.810247 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:36.810232 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bg9p7" podUID="27a3595e-6df7-4999-8257-5c111b074330" Jul 02 21:36:36.810357 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:36.810050 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zp6sf" Jul 02 21:36:36.810357 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:36.810307 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zp6sf" podUID="0bc3fefa-dff5-4181-a239-c6573b8b4356" Jul 02 21:36:37.413739 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:37.413706 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/27a3595e-6df7-4999-8257-5c111b074330-original-pull-secret\") pod \"global-pull-secret-syncer-bg9p7\" (UID: \"27a3595e-6df7-4999-8257-5c111b074330\") " pod="kube-system/global-pull-secret-syncer-bg9p7" Jul 02 21:36:37.414175 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:37.413838 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 21:36:37.414175 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:37.413919 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27a3595e-6df7-4999-8257-5c111b074330-original-pull-secret podName:27a3595e-6df7-4999-8257-5c111b074330 nodeName:}" failed. No retries permitted until 2026-07-02 21:37:09.413900228 +0000 UTC m=+66.133249753 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/27a3595e-6df7-4999-8257-5c111b074330-original-pull-secret") pod "global-pull-secret-syncer-bg9p7" (UID: "27a3595e-6df7-4999-8257-5c111b074330") : object "kube-system"/"original-pull-secret" not registered Jul 02 21:36:37.514068 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:37.514035 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bc3fefa-dff5-4181-a239-c6573b8b4356-metrics-certs\") pod \"network-metrics-daemon-zp6sf\" (UID: \"0bc3fefa-dff5-4181-a239-c6573b8b4356\") " pod="openshift-multus/network-metrics-daemon-zp6sf" Jul 02 21:36:37.514221 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:37.514179 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:36:37.514267 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:37.514234 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bc3fefa-dff5-4181-a239-c6573b8b4356-metrics-certs podName:0bc3fefa-dff5-4181-a239-c6573b8b4356 nodeName:}" failed. No retries permitted until 2026-07-02 21:37:09.514219828 +0000 UTC m=+66.233569347 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0bc3fefa-dff5-4181-a239-c6573b8b4356-metrics-certs") pod "network-metrics-daemon-zp6sf" (UID: "0bc3fefa-dff5-4181-a239-c6573b8b4356") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:36:37.614699 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:37.614667 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnqz8\" (UniqueName: \"kubernetes.io/projected/298fd481-12c6-4869-a4d4-9b17c492706a-kube-api-access-cnqz8\") pod \"network-check-target-zkjd5\" (UID: \"298fd481-12c6-4869-a4d4-9b17c492706a\") " pod="openshift-network-diagnostics/network-check-target-zkjd5" Jul 02 21:36:37.614839 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:37.614820 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 21:36:37.614926 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:37.614843 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 21:36:37.614926 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:37.614854 2556 projected.go:196] Error preparing data for projected volume kube-api-access-cnqz8 for pod openshift-network-diagnostics/network-check-target-zkjd5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:36:37.615007 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:37.614927 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/298fd481-12c6-4869-a4d4-9b17c492706a-kube-api-access-cnqz8 podName:298fd481-12c6-4869-a4d4-9b17c492706a nodeName:}" failed. No retries permitted until 2026-07-02 21:37:09.614911883 +0000 UTC m=+66.334261674 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-cnqz8" (UniqueName: "kubernetes.io/projected/298fd481-12c6-4869-a4d4-9b17c492706a-kube-api-access-cnqz8") pod "network-check-target-zkjd5" (UID: "298fd481-12c6-4869-a4d4-9b17c492706a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:36:38.810624 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:38.810415 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg9p7" Jul 02 21:36:38.811066 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:38.810697 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bg9p7" podUID="27a3595e-6df7-4999-8257-5c111b074330" Jul 02 21:36:38.811066 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:38.810463 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkjd5" Jul 02 21:36:38.811066 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:38.810774 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkjd5" podUID="298fd481-12c6-4869-a4d4-9b17c492706a" Jul 02 21:36:38.811066 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:38.810455 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zp6sf" Jul 02 21:36:38.811066 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:38.810849 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zp6sf" podUID="0bc3fefa-dff5-4181-a239-c6573b8b4356" Jul 02 21:36:40.016003 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.015970 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-146.ec2.internal" event="NodeReady" Jul 02 21:36:40.016448 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.016090 2556 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 02 21:36:40.061659 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.061625 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-m7c4n"] Jul 02 21:36:40.063487 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.063470 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qcb9v"] Jul 02 21:36:40.063589 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.063573 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-m7c4n" Jul 02 21:36:40.065149 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.065132 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qcb9v" Jul 02 21:36:40.065928 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.065911 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 02 21:36:40.066026 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.065944 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 02 21:36:40.066026 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.065991 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bv4sq\"" Jul 02 21:36:40.068144 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.068122 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 02 21:36:40.068239 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.068164 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 02 21:36:40.068239 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.068167 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 02 21:36:40.068386 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.068256 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-nsr8n\"" Jul 02 21:36:40.076043 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.075888 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qcb9v"] Jul 02 21:36:40.076567 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.076549 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-m7c4n"] Jul 02 21:36:40.136483 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.136452 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lmzg\" (UniqueName: \"kubernetes.io/projected/487a1265-7879-4ca9-a747-40bbdba201c4-kube-api-access-6lmzg\") pod \"dns-default-m7c4n\" (UID: \"487a1265-7879-4ca9-a747-40bbdba201c4\") " pod="openshift-dns/dns-default-m7c4n" Jul 02 21:36:40.136591 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.136492 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/487a1265-7879-4ca9-a747-40bbdba201c4-config-volume\") pod \"dns-default-m7c4n\" (UID: \"487a1265-7879-4ca9-a747-40bbdba201c4\") " pod="openshift-dns/dns-default-m7c4n" Jul 02 21:36:40.136591 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.136516 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5a8c260d-9be7-40ca-a29b-edb3cf420235-cert\") pod \"ingress-canary-qcb9v\" (UID: \"5a8c260d-9be7-40ca-a29b-edb3cf420235\") " pod="openshift-ingress-canary/ingress-canary-qcb9v" Jul 02 21:36:40.136591 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.136579 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/487a1265-7879-4ca9-a747-40bbdba201c4-tmp-dir\") pod \"dns-default-m7c4n\" (UID: \"487a1265-7879-4ca9-a747-40bbdba201c4\") " pod="openshift-dns/dns-default-m7c4n" Jul 02 21:36:40.136712 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.136633 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/487a1265-7879-4ca9-a747-40bbdba201c4-metrics-tls\") pod \"dns-default-m7c4n\" (UID: \"487a1265-7879-4ca9-a747-40bbdba201c4\") " pod="openshift-dns/dns-default-m7c4n" Jul 02 21:36:40.136712 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.136670 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c59mn\" (UniqueName: \"kubernetes.io/projected/5a8c260d-9be7-40ca-a29b-edb3cf420235-kube-api-access-c59mn\") pod \"ingress-canary-qcb9v\" (UID: \"5a8c260d-9be7-40ca-a29b-edb3cf420235\") " pod="openshift-ingress-canary/ingress-canary-qcb9v" Jul 02 21:36:40.237512 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.237486 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/487a1265-7879-4ca9-a747-40bbdba201c4-metrics-tls\") pod \"dns-default-m7c4n\" (UID: \"487a1265-7879-4ca9-a747-40bbdba201c4\") " pod="openshift-dns/dns-default-m7c4n" Jul 02 21:36:40.237748 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.237535 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c59mn\" (UniqueName: \"kubernetes.io/projected/5a8c260d-9be7-40ca-a29b-edb3cf420235-kube-api-access-c59mn\") pod \"ingress-canary-qcb9v\" (UID: \"5a8c260d-9be7-40ca-a29b-edb3cf420235\") " pod="openshift-ingress-canary/ingress-canary-qcb9v" Jul 02 21:36:40.237748 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.237576 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lmzg\" (UniqueName: \"kubernetes.io/projected/487a1265-7879-4ca9-a747-40bbdba201c4-kube-api-access-6lmzg\") pod \"dns-default-m7c4n\" (UID: \"487a1265-7879-4ca9-a747-40bbdba201c4\") " pod="openshift-dns/dns-default-m7c4n" Jul 02 21:36:40.237748 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.237598 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/487a1265-7879-4ca9-a747-40bbdba201c4-config-volume\") pod \"dns-default-m7c4n\" (UID: \"487a1265-7879-4ca9-a747-40bbdba201c4\") " pod="openshift-dns/dns-default-m7c4n" Jul 02 21:36:40.237748 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.237619 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5a8c260d-9be7-40ca-a29b-edb3cf420235-cert\") pod \"ingress-canary-qcb9v\" (UID: \"5a8c260d-9be7-40ca-a29b-edb3cf420235\") " pod="openshift-ingress-canary/ingress-canary-qcb9v" Jul 02 21:36:40.237748 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.237652 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/487a1265-7879-4ca9-a747-40bbdba201c4-tmp-dir\") pod \"dns-default-m7c4n\" (UID: \"487a1265-7879-4ca9-a747-40bbdba201c4\") " pod="openshift-dns/dns-default-m7c4n" Jul 02 21:36:40.238045 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:40.238024 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 21:36:40.238097 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:40.238091 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/487a1265-7879-4ca9-a747-40bbdba201c4-metrics-tls podName:487a1265-7879-4ca9-a747-40bbdba201c4 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:40.738073519 +0000 UTC m=+37.457423066 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/487a1265-7879-4ca9-a747-40bbdba201c4-metrics-tls") pod "dns-default-m7c4n" (UID: "487a1265-7879-4ca9-a747-40bbdba201c4") : secret "dns-default-metrics-tls" not found Jul 02 21:36:40.238200 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.238181 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/487a1265-7879-4ca9-a747-40bbdba201c4-config-volume\") pod \"dns-default-m7c4n\" (UID: \"487a1265-7879-4ca9-a747-40bbdba201c4\") " pod="openshift-dns/dns-default-m7c4n" Jul 02 21:36:40.238247 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.238025 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/487a1265-7879-4ca9-a747-40bbdba201c4-tmp-dir\") pod \"dns-default-m7c4n\" (UID: \"487a1265-7879-4ca9-a747-40bbdba201c4\") " pod="openshift-dns/dns-default-m7c4n" Jul 02 21:36:40.242067 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.242043 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5a8c260d-9be7-40ca-a29b-edb3cf420235-cert\") pod \"ingress-canary-qcb9v\" (UID: \"5a8c260d-9be7-40ca-a29b-edb3cf420235\") " pod="openshift-ingress-canary/ingress-canary-qcb9v" Jul 02 21:36:40.246778 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.246749 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lmzg\" (UniqueName: \"kubernetes.io/projected/487a1265-7879-4ca9-a747-40bbdba201c4-kube-api-access-6lmzg\") pod \"dns-default-m7c4n\" (UID: \"487a1265-7879-4ca9-a747-40bbdba201c4\") " pod="openshift-dns/dns-default-m7c4n" Jul 02 21:36:40.247065 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.247042 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c59mn\" (UniqueName: \"kubernetes.io/projected/5a8c260d-9be7-40ca-a29b-edb3cf420235-kube-api-access-c59mn\") pod \"ingress-canary-qcb9v\" (UID: \"5a8c260d-9be7-40ca-a29b-edb3cf420235\") " pod="openshift-ingress-canary/ingress-canary-qcb9v" Jul 02 21:36:40.380186 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.380143 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qcb9v" Jul 02 21:36:40.511322 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.511288 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qcb9v"] Jul 02 21:36:40.516211 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:40.516160 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a8c260d_9be7_40ca_a29b_edb3cf420235.slice/crio-7f58c0543c9f26afc05ffec3da82234dd4adf845fc29e58eaa43aac01b2c1031 WatchSource:0}: Error finding container 7f58c0543c9f26afc05ffec3da82234dd4adf845fc29e58eaa43aac01b2c1031: Status 404 returned error can't find the container with id 7f58c0543c9f26afc05ffec3da82234dd4adf845fc29e58eaa43aac01b2c1031 Jul 02 21:36:40.742295 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.742251 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/487a1265-7879-4ca9-a747-40bbdba201c4-metrics-tls\") pod \"dns-default-m7c4n\" (UID: \"487a1265-7879-4ca9-a747-40bbdba201c4\") " pod="openshift-dns/dns-default-m7c4n" Jul 02 21:36:40.742471 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:40.742410 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 21:36:40.742510 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:40.742478 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/487a1265-7879-4ca9-a747-40bbdba201c4-metrics-tls podName:487a1265-7879-4ca9-a747-40bbdba201c4 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:41.742463607 +0000 UTC m=+38.461813129 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/487a1265-7879-4ca9-a747-40bbdba201c4-metrics-tls") pod "dns-default-m7c4n" (UID: "487a1265-7879-4ca9-a747-40bbdba201c4") : secret "dns-default-metrics-tls" not found Jul 02 21:36:40.810846 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.810779 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkjd5" Jul 02 21:36:40.810970 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.810779 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg9p7" Jul 02 21:36:40.811020 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.810779 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zp6sf" Jul 02 21:36:40.813986 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.813971 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 21:36:40.814050 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.814008 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 21:36:40.815241 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.815227 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 21:36:40.815321 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.815306 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 21:36:40.815365 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.815317 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dkhhs\"" Jul 02 21:36:40.815365 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:40.815326 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ckjxv\"" Jul 02 21:36:41.022610 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:41.022558 2556 generic.go:356] "Generic (PLEG): container finished" podID="eb1cecd6-d74b-4f21-bb0a-02f534221301" containerID="2b136567770fbd74b1fc604ed5f9c26779cc79173324c1de430ba6b5dd7ceb45" exitCode=0 Jul 02 21:36:41.023003 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:41.022673 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qmwz4" event={"ID":"eb1cecd6-d74b-4f21-bb0a-02f534221301","Type":"ContainerDied","Data":"2b136567770fbd74b1fc604ed5f9c26779cc79173324c1de430ba6b5dd7ceb45"} Jul 02 21:36:41.024053 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:41.024017 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qcb9v" event={"ID":"5a8c260d-9be7-40ca-a29b-edb3cf420235","Type":"ContainerStarted","Data":"7f58c0543c9f26afc05ffec3da82234dd4adf845fc29e58eaa43aac01b2c1031"} Jul 02 21:36:41.359557 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:41.359274 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:41.443935 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:41.443908 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:41.751118 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:41.751027 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/487a1265-7879-4ca9-a747-40bbdba201c4-metrics-tls\") pod \"dns-default-m7c4n\" (UID: \"487a1265-7879-4ca9-a747-40bbdba201c4\") " pod="openshift-dns/dns-default-m7c4n" Jul 02 21:36:41.751314 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:41.751194 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 21:36:41.751314 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:41.751265 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/487a1265-7879-4ca9-a747-40bbdba201c4-metrics-tls podName:487a1265-7879-4ca9-a747-40bbdba201c4 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:43.75124364 +0000 UTC m=+40.470593165 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/487a1265-7879-4ca9-a747-40bbdba201c4-metrics-tls") pod "dns-default-m7c4n" (UID: "487a1265-7879-4ca9-a747-40bbdba201c4") : secret "dns-default-metrics-tls" not found Jul 02 21:36:42.010812 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:42.010774 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:42.030275 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:42.030250 2556 generic.go:356] "Generic (PLEG): container finished" podID="eb1cecd6-d74b-4f21-bb0a-02f534221301" containerID="13f8e1fbe894797f1cc104538d2fcaaca982f3ca8355b275e1d25a4d15a0b4e6" exitCode=0 Jul 02 21:36:42.030657 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:42.030297 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qmwz4" event={"ID":"eb1cecd6-d74b-4f21-bb0a-02f534221301","Type":"ContainerDied","Data":"13f8e1fbe894797f1cc104538d2fcaaca982f3ca8355b275e1d25a4d15a0b4e6"} Jul 02 21:36:42.079405 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:42.079377 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:43.034557 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:43.034532 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qmwz4" event={"ID":"eb1cecd6-d74b-4f21-bb0a-02f534221301","Type":"ContainerStarted","Data":"170aee9acfbf87b2823214d09e50a19e980eeb2fe81416d650605cb6bff0f776"} Jul 02 21:36:43.065283 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:43.065164 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qmwz4" podStartSLOduration=10.474111503 podStartE2EDuration="40.06514483s" totalImagesPullingTime="29.591033327s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 21:36:03 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:36:43.063396033 +0000 UTC m=+39.782745573" watchObservedRunningTime="2026-07-02 21:36:43.06514483 +0000 UTC m=+39.784494373" Jul 02 21:36:43.184024 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:43.183995 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:43.242484 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:43.242445 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:43.525465 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:43.525439 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:43.767583 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:43.767555 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/487a1265-7879-4ca9-a747-40bbdba201c4-metrics-tls\") pod \"dns-default-m7c4n\" (UID: \"487a1265-7879-4ca9-a747-40bbdba201c4\") " pod="openshift-dns/dns-default-m7c4n" Jul 02 21:36:43.767710 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:43.767676 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 21:36:43.767764 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:43.767722 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/487a1265-7879-4ca9-a747-40bbdba201c4-metrics-tls podName:487a1265-7879-4ca9-a747-40bbdba201c4 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:47.767710596 +0000 UTC m=+44.487060114 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/487a1265-7879-4ca9-a747-40bbdba201c4-metrics-tls") pod "dns-default-m7c4n" (UID: "487a1265-7879-4ca9-a747-40bbdba201c4") : secret "dns-default-metrics-tls" not found Jul 02 21:36:44.038166 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:44.038128 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qcb9v" event={"ID":"5a8c260d-9be7-40ca-a29b-edb3cf420235","Type":"ContainerStarted","Data":"366fbbc50057f69040ea5b8297fb5425924a879fbf7f187b948abbaec0315383"} Jul 02 21:36:44.054145 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:44.054106 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qcb9v" podStartSLOduration=1.547929167 podStartE2EDuration="4.054091813s" totalImagesPullingTime="2.506162646s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:36:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:36:44.053361095 +0000 UTC m=+40.772710635" watchObservedRunningTime="2026-07-02 21:36:44.054091813 +0000 UTC m=+40.773441353" Jul 02 21:36:44.450574 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:44.450506 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:44.507370 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:44.507339 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:46.633090 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:46.633057 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:46.689309 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:46.689281 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:46.965455 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:46.965370 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:47.068070 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.068037 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:47.165122 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.165093 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:47.579654 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.579619 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lzqbp"] Jul 02 21:36:47.593957 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.593937 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:47.607206 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.607174 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lzqbp"] Jul 02 21:36:47.607313 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.607282 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lzqbp" Jul 02 21:36:47.610059 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.610033 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 02 21:36:47.610170 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.610034 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 02 21:36:47.610170 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.610033 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 02 21:36:47.611134 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.611119 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pbj6p\"" Jul 02 21:36:47.611221 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.611141 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 02 21:36:47.617979 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.617961 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-xb5mh"] Jul 02 21:36:47.643083 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.643065 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-xb5mh"] Jul 02 21:36:47.643369 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.643151 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-xb5mh" Jul 02 21:36:47.645454 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.645439 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 02 21:36:47.645507 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.645455 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-zvn4n\"" Jul 02 21:36:47.645507 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.645482 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 02 21:36:47.667336 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.667309 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:47.694288 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.694268 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/245f2091-5cf4-4185-a0fb-629ec98a0be0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lzqbp\" (UID: \"245f2091-5cf4-4185-a0fb-629ec98a0be0\") " pod="openshift-insights/insights-runtime-extractor-lzqbp" Jul 02 21:36:47.694360 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.694298 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdfjp\" (UniqueName: \"kubernetes.io/projected/245f2091-5cf4-4185-a0fb-629ec98a0be0-kube-api-access-xdfjp\") pod \"insights-runtime-extractor-lzqbp\" (UID: \"245f2091-5cf4-4185-a0fb-629ec98a0be0\") " pod="openshift-insights/insights-runtime-extractor-lzqbp" Jul 02 21:36:47.694360 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.694336 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/245f2091-5cf4-4185-a0fb-629ec98a0be0-crio-socket\") pod \"insights-runtime-extractor-lzqbp\" (UID: \"245f2091-5cf4-4185-a0fb-629ec98a0be0\") " pod="openshift-insights/insights-runtime-extractor-lzqbp" Jul 02 21:36:47.694430 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.694360 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/245f2091-5cf4-4185-a0fb-629ec98a0be0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lzqbp\" (UID: \"245f2091-5cf4-4185-a0fb-629ec98a0be0\") " pod="openshift-insights/insights-runtime-extractor-lzqbp" Jul 02 21:36:47.694430 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.694392 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/245f2091-5cf4-4185-a0fb-629ec98a0be0-data-volume\") pod \"insights-runtime-extractor-lzqbp\" (UID: \"245f2091-5cf4-4185-a0fb-629ec98a0be0\") " pod="openshift-insights/insights-runtime-extractor-lzqbp" Jul 02 21:36:47.719658 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.719638 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:47.795333 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.795310 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbpdc\" (UniqueName: \"kubernetes.io/projected/c96f6156-aad2-4db8-af99-ea5f38b27ed2-kube-api-access-sbpdc\") pod \"migrator-d57d58cc9-xb5mh\" (UID: \"c96f6156-aad2-4db8-af99-ea5f38b27ed2\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-xb5mh" Jul 02 21:36:47.795429 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.795341 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/245f2091-5cf4-4185-a0fb-629ec98a0be0-crio-socket\") pod \"insights-runtime-extractor-lzqbp\" (UID: \"245f2091-5cf4-4185-a0fb-629ec98a0be0\") " pod="openshift-insights/insights-runtime-extractor-lzqbp" Jul 02 21:36:47.795429 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.795365 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/245f2091-5cf4-4185-a0fb-629ec98a0be0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lzqbp\" (UID: \"245f2091-5cf4-4185-a0fb-629ec98a0be0\") " pod="openshift-insights/insights-runtime-extractor-lzqbp" Jul 02 21:36:47.795502 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.795426 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/487a1265-7879-4ca9-a747-40bbdba201c4-metrics-tls\") pod \"dns-default-m7c4n\" (UID: \"487a1265-7879-4ca9-a747-40bbdba201c4\") " pod="openshift-dns/dns-default-m7c4n" Jul 02 21:36:47.795502 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:47.795441 2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 21:36:47.795502 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.795453 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/245f2091-5cf4-4185-a0fb-629ec98a0be0-data-volume\") pod \"insights-runtime-extractor-lzqbp\" (UID: \"245f2091-5cf4-4185-a0fb-629ec98a0be0\") " pod="openshift-insights/insights-runtime-extractor-lzqbp" Jul 02 21:36:47.795597 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:47.795504 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/245f2091-5cf4-4185-a0fb-629ec98a0be0-kube-rbac-proxy-cm podName:245f2091-5cf4-4185-a0fb-629ec98a0be0 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:48.29548726 +0000 UTC m=+45.014836795 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/245f2091-5cf4-4185-a0fb-629ec98a0be0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lzqbp" (UID: "245f2091-5cf4-4185-a0fb-629ec98a0be0") : configmap "kube-rbac-proxy" not found Jul 02 21:36:47.795597 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.795533 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/245f2091-5cf4-4185-a0fb-629ec98a0be0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lzqbp\" (UID: \"245f2091-5cf4-4185-a0fb-629ec98a0be0\") " pod="openshift-insights/insights-runtime-extractor-lzqbp" Jul 02 21:36:47.795597 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.795533 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/245f2091-5cf4-4185-a0fb-629ec98a0be0-crio-socket\") pod \"insights-runtime-extractor-lzqbp\" (UID: \"245f2091-5cf4-4185-a0fb-629ec98a0be0\") " pod="openshift-insights/insights-runtime-extractor-lzqbp" Jul 02 21:36:47.795597 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:47.795563 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 21:36:47.795597 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.795570 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdfjp\" (UniqueName: \"kubernetes.io/projected/245f2091-5cf4-4185-a0fb-629ec98a0be0-kube-api-access-xdfjp\") pod \"insights-runtime-extractor-lzqbp\" (UID: \"245f2091-5cf4-4185-a0fb-629ec98a0be0\") " pod="openshift-insights/insights-runtime-extractor-lzqbp" Jul 02 21:36:47.795761 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:47.795608 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/487a1265-7879-4ca9-a747-40bbdba201c4-metrics-tls podName:487a1265-7879-4ca9-a747-40bbdba201c4 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:55.795595519 +0000 UTC m=+52.514945040 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/487a1265-7879-4ca9-a747-40bbdba201c4-metrics-tls") pod "dns-default-m7c4n" (UID: "487a1265-7879-4ca9-a747-40bbdba201c4") : secret "dns-default-metrics-tls" not found Jul 02 21:36:47.795761 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:47.795665 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 21:36:47.795761 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:47.795700 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/245f2091-5cf4-4185-a0fb-629ec98a0be0-insights-runtime-extractor-tls podName:245f2091-5cf4-4185-a0fb-629ec98a0be0 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:48.295689069 +0000 UTC m=+45.015038590 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/245f2091-5cf4-4185-a0fb-629ec98a0be0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lzqbp" (UID: "245f2091-5cf4-4185-a0fb-629ec98a0be0") : secret "insights-runtime-extractor-tls" not found Jul 02 21:36:47.795915 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.795768 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/245f2091-5cf4-4185-a0fb-629ec98a0be0-data-volume\") pod \"insights-runtime-extractor-lzqbp\" (UID: \"245f2091-5cf4-4185-a0fb-629ec98a0be0\") " pod="openshift-insights/insights-runtime-extractor-lzqbp" Jul 02 21:36:47.806839 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.806821 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdfjp\" (UniqueName: \"kubernetes.io/projected/245f2091-5cf4-4185-a0fb-629ec98a0be0-kube-api-access-xdfjp\") pod \"insights-runtime-extractor-lzqbp\" (UID: \"245f2091-5cf4-4185-a0fb-629ec98a0be0\") " pod="openshift-insights/insights-runtime-extractor-lzqbp" Jul 02 21:36:47.819359 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.819319 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:47.896583 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.896531 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbpdc\" (UniqueName: \"kubernetes.io/projected/c96f6156-aad2-4db8-af99-ea5f38b27ed2-kube-api-access-sbpdc\") pod \"migrator-d57d58cc9-xb5mh\" (UID: \"c96f6156-aad2-4db8-af99-ea5f38b27ed2\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-xb5mh" Jul 02 21:36:47.910422 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.910398 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbpdc\" (UniqueName: \"kubernetes.io/projected/c96f6156-aad2-4db8-af99-ea5f38b27ed2-kube-api-access-sbpdc\") pod \"migrator-d57d58cc9-xb5mh\" (UID: \"c96f6156-aad2-4db8-af99-ea5f38b27ed2\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-xb5mh" Jul 02 21:36:47.950340 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.950316 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-xb5mh" Jul 02 21:36:47.985157 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:47.985128 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:48.043834 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:48.043807 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:48.092488 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:48.092462 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-xb5mh"] Jul 02 21:36:48.095432 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:36:48.095404 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc96f6156_aad2_4db8_af99_ea5f38b27ed2.slice/crio-6c6009bdfd9a5865df0bb7b10c08d0e8450678b63c8e36dc628501c6a076720e WatchSource:0}: Error finding container 6c6009bdfd9a5865df0bb7b10c08d0e8450678b63c8e36dc628501c6a076720e: Status 404 returned error can't find the container with id 6c6009bdfd9a5865df0bb7b10c08d0e8450678b63c8e36dc628501c6a076720e Jul 02 21:36:48.300067 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:48.300044 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/245f2091-5cf4-4185-a0fb-629ec98a0be0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lzqbp\" (UID: \"245f2091-5cf4-4185-a0fb-629ec98a0be0\") " pod="openshift-insights/insights-runtime-extractor-lzqbp" Jul 02 21:36:48.300186 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:48.300112 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/245f2091-5cf4-4185-a0fb-629ec98a0be0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lzqbp\" (UID: \"245f2091-5cf4-4185-a0fb-629ec98a0be0\") " pod="openshift-insights/insights-runtime-extractor-lzqbp" Jul 02 21:36:48.300186 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:48.300169 2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 21:36:48.300251 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:48.300220 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 21:36:48.300251 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:48.300241 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/245f2091-5cf4-4185-a0fb-629ec98a0be0-kube-rbac-proxy-cm podName:245f2091-5cf4-4185-a0fb-629ec98a0be0 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:49.300222416 +0000 UTC m=+46.019571956 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/245f2091-5cf4-4185-a0fb-629ec98a0be0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lzqbp" (UID: "245f2091-5cf4-4185-a0fb-629ec98a0be0") : configmap "kube-rbac-proxy" not found Jul 02 21:36:48.300320 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:48.300259 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/245f2091-5cf4-4185-a0fb-629ec98a0be0-insights-runtime-extractor-tls podName:245f2091-5cf4-4185-a0fb-629ec98a0be0 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:49.300248763 +0000 UTC m=+46.019598282 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/245f2091-5cf4-4185-a0fb-629ec98a0be0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lzqbp" (UID: "245f2091-5cf4-4185-a0fb-629ec98a0be0") : secret "insights-runtime-extractor-tls" not found Jul 02 21:36:48.465188 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:48.465142 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:48.516361 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:48.516333 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:49.051448 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:49.051413 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-xb5mh" event={"ID":"c96f6156-aad2-4db8-af99-ea5f38b27ed2","Type":"ContainerStarted","Data":"6c6009bdfd9a5865df0bb7b10c08d0e8450678b63c8e36dc628501c6a076720e"} Jul 02 21:36:49.308525 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:49.308447 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/245f2091-5cf4-4185-a0fb-629ec98a0be0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lzqbp\" (UID: \"245f2091-5cf4-4185-a0fb-629ec98a0be0\") " pod="openshift-insights/insights-runtime-extractor-lzqbp" Jul 02 21:36:49.308718 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:49.308555 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/245f2091-5cf4-4185-a0fb-629ec98a0be0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lzqbp\" (UID: \"245f2091-5cf4-4185-a0fb-629ec98a0be0\") " pod="openshift-insights/insights-runtime-extractor-lzqbp" Jul 02 21:36:49.308718 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:49.308566 2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 21:36:49.308718 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:49.308644 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/245f2091-5cf4-4185-a0fb-629ec98a0be0-kube-rbac-proxy-cm podName:245f2091-5cf4-4185-a0fb-629ec98a0be0 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:51.308620806 +0000 UTC m=+48.027970330 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/245f2091-5cf4-4185-a0fb-629ec98a0be0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lzqbp" (UID: "245f2091-5cf4-4185-a0fb-629ec98a0be0") : configmap "kube-rbac-proxy" not found Jul 02 21:36:49.308718 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:49.308656 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 21:36:49.308718 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:49.308699 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/245f2091-5cf4-4185-a0fb-629ec98a0be0-insights-runtime-extractor-tls podName:245f2091-5cf4-4185-a0fb-629ec98a0be0 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:51.308685184 +0000 UTC m=+48.028034704 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/245f2091-5cf4-4185-a0fb-629ec98a0be0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lzqbp" (UID: "245f2091-5cf4-4185-a0fb-629ec98a0be0") : secret "insights-runtime-extractor-tls" not found Jul 02 21:36:49.748948 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:49.748860 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jgmz2_222d3332-3472-479e-a8e2-8d8d9c702a9b/dns-node-resolver/0.log" Jul 02 21:36:50.055156 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:50.055133 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-xb5mh" event={"ID":"c96f6156-aad2-4db8-af99-ea5f38b27ed2","Type":"ContainerStarted","Data":"a88093442ffe560ef2978ef48ec13e40c97484d655e86fe24d2c45a29dd519b5"} Jul 02 21:36:50.163036 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:50.163003 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:50.263420 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:50.263393 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:50.549747 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:50.549722 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-82ctv_baf560bd-eaae-468e-bc1c-a007fe68bded/node-ca/0.log" Jul 02 21:36:51.059270 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:51.059238 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-xb5mh" event={"ID":"c96f6156-aad2-4db8-af99-ea5f38b27ed2","Type":"ContainerStarted","Data":"39db6132f8cc41651f00642f8d9ce86ab8f2909f86ea04cebedf71006cf71d79"} Jul 02 21:36:51.062617 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:51.062597 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:51.077941 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:51.077900 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-xb5mh" podStartSLOduration=2.319161343 podStartE2EDuration="4.077887283s" totalImagesPullingTime="1.75872594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:36:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:36:51.07670295 +0000 UTC m=+47.796052490" watchObservedRunningTime="2026-07-02 21:36:51.077887283 +0000 UTC m=+47.797236815" Jul 02 21:36:51.321942 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:51.321861 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/245f2091-5cf4-4185-a0fb-629ec98a0be0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lzqbp\" (UID: \"245f2091-5cf4-4185-a0fb-629ec98a0be0\") " pod="openshift-insights/insights-runtime-extractor-lzqbp" Jul 02 21:36:51.322058 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:51.321965 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/245f2091-5cf4-4185-a0fb-629ec98a0be0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lzqbp\" (UID: \"245f2091-5cf4-4185-a0fb-629ec98a0be0\") " pod="openshift-insights/insights-runtime-extractor-lzqbp" Jul 02 21:36:51.322058 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:51.321978 2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 21:36:51.322058 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:51.322036 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/245f2091-5cf4-4185-a0fb-629ec98a0be0-kube-rbac-proxy-cm podName:245f2091-5cf4-4185-a0fb-629ec98a0be0 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:55.322022535 +0000 UTC m=+52.041372054 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/245f2091-5cf4-4185-a0fb-629ec98a0be0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lzqbp" (UID: "245f2091-5cf4-4185-a0fb-629ec98a0be0") : configmap "kube-rbac-proxy" not found Jul 02 21:36:51.322156 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:51.322073 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 21:36:51.322156 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:51.322119 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/245f2091-5cf4-4185-a0fb-629ec98a0be0-insights-runtime-extractor-tls podName:245f2091-5cf4-4185-a0fb-629ec98a0be0 nodeName:}" failed. No retries permitted until 2026-07-02 21:36:55.322105524 +0000 UTC m=+52.041455070 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/245f2091-5cf4-4185-a0fb-629ec98a0be0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lzqbp" (UID: "245f2091-5cf4-4185-a0fb-629ec98a0be0") : secret "insights-runtime-extractor-tls" not found Jul 02 21:36:51.348794 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:51.348775 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qcb9v_5a8c260d-9be7-40ca-a29b-edb3cf420235/serve-healthcheck-canary/0.log" Jul 02 21:36:52.388411 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:52.388383 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:53.346987 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:53.346956 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:53.537942 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:53.537911 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:53.638835 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:53.638767 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:53.738361 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:53.738332 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:53.838419 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:53.838392 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:53.941914 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:53.941834 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:54.039811 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:54.039789 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:54.632136 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:54.632110 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:55.036216 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:55.036189 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:55.349376 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:55.349299 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/245f2091-5cf4-4185-a0fb-629ec98a0be0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lzqbp\" (UID: \"245f2091-5cf4-4185-a0fb-629ec98a0be0\") " pod="openshift-insights/insights-runtime-extractor-lzqbp" Jul 02 21:36:55.349376 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:55.349368 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/245f2091-5cf4-4185-a0fb-629ec98a0be0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lzqbp\" (UID: \"245f2091-5cf4-4185-a0fb-629ec98a0be0\") " pod="openshift-insights/insights-runtime-extractor-lzqbp" Jul 02 21:36:55.349552 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:55.349467 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 21:36:55.349552 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:55.349528 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/245f2091-5cf4-4185-a0fb-629ec98a0be0-insights-runtime-extractor-tls podName:245f2091-5cf4-4185-a0fb-629ec98a0be0 nodeName:}" failed. No retries permitted until 2026-07-02 21:37:03.349514268 +0000 UTC m=+60.068863788 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/245f2091-5cf4-4185-a0fb-629ec98a0be0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lzqbp" (UID: "245f2091-5cf4-4185-a0fb-629ec98a0be0") : secret "insights-runtime-extractor-tls" not found Jul 02 21:36:55.349856 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:55.349836 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/245f2091-5cf4-4185-a0fb-629ec98a0be0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lzqbp\" (UID: \"245f2091-5cf4-4185-a0fb-629ec98a0be0\") " pod="openshift-insights/insights-runtime-extractor-lzqbp" Jul 02 21:36:55.425669 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:55.425635 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:55.631213 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:55.631147 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:55.854237 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:55.854207 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/487a1265-7879-4ca9-a747-40bbdba201c4-metrics-tls\") pod \"dns-default-m7c4n\" (UID: \"487a1265-7879-4ca9-a747-40bbdba201c4\") " pod="openshift-dns/dns-default-m7c4n" Jul 02 21:36:55.854634 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:55.854308 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 21:36:55.854634 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:36:55.854354 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/487a1265-7879-4ca9-a747-40bbdba201c4-metrics-tls podName:487a1265-7879-4ca9-a747-40bbdba201c4 nodeName:}" failed. No retries permitted until 2026-07-02 21:37:11.854342504 +0000 UTC m=+68.573692025 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/487a1265-7879-4ca9-a747-40bbdba201c4-metrics-tls") pod "dns-default-m7c4n" (UID: "487a1265-7879-4ca9-a747-40bbdba201c4") : secret "dns-default-metrics-tls" not found Jul 02 21:36:56.232553 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:56.232523 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:56.424744 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:56.424710 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:56.831390 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:56.831353 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:57.227185 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:57.227108 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:57.437018 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:57.436984 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:57.491146 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:57.491076 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:58.025540 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:58.025509 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:58.222613 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:58.222578 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:58.634094 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:58.634064 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:58.822094 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:58.822032 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:59.226543 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:59.226508 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:59.818771 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:59.818744 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:36:59.869656 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:36:59.869631 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:00.222082 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:00.222002 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:00.357690 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:00.357662 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:00.458549 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:00.458528 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:00.622211 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:00.622186 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:01.023778 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:01.023745 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:01.431898 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:01.431796 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:01.622770 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:01.622743 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:02.040092 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:02.040056 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:02.626815 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:02.626781 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:02.742049 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:02.742009 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:03.026905 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:03.026823 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:03.410258 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:03.410173 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/245f2091-5cf4-4185-a0fb-629ec98a0be0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lzqbp\" (UID: \"245f2091-5cf4-4185-a0fb-629ec98a0be0\") " pod="openshift-insights/insights-runtime-extractor-lzqbp" Jul 02 21:37:03.412572 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:03.412553 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/245f2091-5cf4-4185-a0fb-629ec98a0be0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lzqbp\" (UID: \"245f2091-5cf4-4185-a0fb-629ec98a0be0\") " pod="openshift-insights/insights-runtime-extractor-lzqbp" Jul 02 21:37:03.427196 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:03.427159 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:03.516623 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:03.516592 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lzqbp" Jul 02 21:37:03.631280 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:03.631164 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:03.678207 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:03.678141 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lzqbp"] Jul 02 21:37:03.681296 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:37:03.681261 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod245f2091_5cf4_4185_a0fb_629ec98a0be0.slice/crio-73b456cad8b8f0821e56ce5fc8dbbd1a822173223c08cc98d4ea26f8a8f68d4d WatchSource:0}: Error finding container 73b456cad8b8f0821e56ce5fc8dbbd1a822173223c08cc98d4ea26f8a8f68d4d: Status 404 returned error can't find the container with id 73b456cad8b8f0821e56ce5fc8dbbd1a822173223c08cc98d4ea26f8a8f68d4d Jul 02 21:37:04.018989 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:04.018962 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s8jx8" Jul 02 21:37:04.033852 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:04.033824 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:04.089894 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:04.089766 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lzqbp" event={"ID":"245f2091-5cf4-4185-a0fb-629ec98a0be0","Type":"ContainerStarted","Data":"73b456cad8b8f0821e56ce5fc8dbbd1a822173223c08cc98d4ea26f8a8f68d4d"} Jul 02 21:37:04.232653 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:04.232612 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:08.104330 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:08.104301 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lzqbp" event={"ID":"245f2091-5cf4-4185-a0fb-629ec98a0be0","Type":"ContainerStarted","Data":"cc43f5f076ce3d4c9b27ebe73afa22e0d39def188e4384f384e339e66e5c156d"} Jul 02 21:37:08.104596 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:08.104359 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lzqbp" event={"ID":"245f2091-5cf4-4185-a0fb-629ec98a0be0","Type":"ContainerStarted","Data":"63d527994fca0ad4a5765ea5d29ade2ae24a2920a7b08174f73e7b5c535f7655"} Jul 02 21:37:09.177344 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.177303 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b9cc8b67b-hxfmp"] Jul 02 21:37:09.181462 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.181417 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86b9db775-lkwv2"] Jul 02 21:37:09.181607 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.181562 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b9cc8b67b-hxfmp" Jul 02 21:37:09.184002 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.183981 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86b9db775-lkwv2" Jul 02 21:37:09.184152 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.184133 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 02 21:37:09.184266 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.184247 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 02 21:37:09.184309 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.184297 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-hfmqv\"" Jul 02 21:37:09.185147 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.185129 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 02 21:37:09.185345 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.185329 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 02 21:37:09.186552 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.186311 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 02 21:37:09.186719 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.186702 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 02 21:37:09.186967 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.186949 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 02 21:37:09.187411 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.187397 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 02 21:37:09.195035 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.195016 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b9cc8b67b-hxfmp"] Jul 02 21:37:09.204155 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.204139 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86b9db775-lkwv2"] Jul 02 21:37:09.257266 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.257236 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxtpg\" (UniqueName: \"kubernetes.io/projected/3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9-kube-api-access-cxtpg\") pod \"cluster-proxy-proxy-agent-86b9db775-lkwv2\" (UID: \"3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86b9db775-lkwv2" Jul 02 21:37:09.257416 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.257285 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9-hub\") pod \"cluster-proxy-proxy-agent-86b9db775-lkwv2\" (UID: \"3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86b9db775-lkwv2" Jul 02 21:37:09.257416 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.257314 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-86b9db775-lkwv2\" (UID: \"3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86b9db775-lkwv2" Jul 02 21:37:09.257416 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.257389 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-86b9db775-lkwv2\" (UID: \"3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86b9db775-lkwv2" Jul 02 21:37:09.257558 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.257421 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-455nx\" (UniqueName: \"kubernetes.io/projected/a0ce982a-5708-426c-b332-2271f0d81c22-kube-api-access-455nx\") pod \"managed-serviceaccount-addon-agent-7b9cc8b67b-hxfmp\" (UID: \"a0ce982a-5708-426c-b332-2271f0d81c22\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b9cc8b67b-hxfmp" Jul 02 21:37:09.257558 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.257457 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a0ce982a-5708-426c-b332-2271f0d81c22-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7b9cc8b67b-hxfmp\" (UID: \"a0ce982a-5708-426c-b332-2271f0d81c22\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b9cc8b67b-hxfmp" Jul 02 21:37:09.257558 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.257485 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9-ca\") pod \"cluster-proxy-proxy-agent-86b9db775-lkwv2\" (UID: \"3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86b9db775-lkwv2" Jul 02 21:37:09.257558 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.257499 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-86b9db775-lkwv2\" (UID: \"3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86b9db775-lkwv2" Jul 02 21:37:09.269604 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.269577 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:09.347070 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.347033 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:09.358276 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.358251 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a0ce982a-5708-426c-b332-2271f0d81c22-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7b9cc8b67b-hxfmp\" (UID: \"a0ce982a-5708-426c-b332-2271f0d81c22\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b9cc8b67b-hxfmp" Jul 02 21:37:09.358426 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.358297 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9-ca\") pod \"cluster-proxy-proxy-agent-86b9db775-lkwv2\" (UID: \"3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86b9db775-lkwv2" Jul 02 21:37:09.358426 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.358324 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-86b9db775-lkwv2\" (UID: \"3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86b9db775-lkwv2" Jul 02 21:37:09.358426 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.358374 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxtpg\" (UniqueName: \"kubernetes.io/projected/3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9-kube-api-access-cxtpg\") pod \"cluster-proxy-proxy-agent-86b9db775-lkwv2\" (UID: \"3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86b9db775-lkwv2" Jul 02 21:37:09.358580 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.358509 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9-hub\") pod \"cluster-proxy-proxy-agent-86b9db775-lkwv2\" (UID: \"3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86b9db775-lkwv2" Jul 02 21:37:09.358580 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.358548 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-86b9db775-lkwv2\" (UID: \"3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86b9db775-lkwv2" Jul 02 21:37:09.358675 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.358595 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-86b9db775-lkwv2\" (UID: \"3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86b9db775-lkwv2" Jul 02 21:37:09.358675 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.358631 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-455nx\" (UniqueName: \"kubernetes.io/projected/a0ce982a-5708-426c-b332-2271f0d81c22-kube-api-access-455nx\") pod \"managed-serviceaccount-addon-agent-7b9cc8b67b-hxfmp\" (UID: \"a0ce982a-5708-426c-b332-2271f0d81c22\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b9cc8b67b-hxfmp" Jul 02 21:37:09.359541 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.359515 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-86b9db775-lkwv2\" (UID: \"3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86b9db775-lkwv2" Jul 02 21:37:09.361466 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.361446 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9-ca\") pod \"cluster-proxy-proxy-agent-86b9db775-lkwv2\" (UID: \"3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86b9db775-lkwv2" Jul 02 21:37:09.361575 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.361461 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-86b9db775-lkwv2\" (UID: \"3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86b9db775-lkwv2" Jul 02 21:37:09.361628 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.361613 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9-hub\") pod \"cluster-proxy-proxy-agent-86b9db775-lkwv2\" (UID: \"3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86b9db775-lkwv2" Jul 02 21:37:09.361857 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.361842 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-86b9db775-lkwv2\" (UID: \"3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86b9db775-lkwv2" Jul 02 21:37:09.362016 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.361993 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a0ce982a-5708-426c-b332-2271f0d81c22-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7b9cc8b67b-hxfmp\" (UID: \"a0ce982a-5708-426c-b332-2271f0d81c22\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b9cc8b67b-hxfmp" Jul 02 21:37:09.369672 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.369648 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-455nx\" (UniqueName: \"kubernetes.io/projected/a0ce982a-5708-426c-b332-2271f0d81c22-kube-api-access-455nx\") pod \"managed-serviceaccount-addon-agent-7b9cc8b67b-hxfmp\" (UID: \"a0ce982a-5708-426c-b332-2271f0d81c22\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b9cc8b67b-hxfmp" Jul 02 21:37:09.370196 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.370170 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxtpg\" (UniqueName: \"kubernetes.io/projected/3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9-kube-api-access-cxtpg\") pod \"cluster-proxy-proxy-agent-86b9db775-lkwv2\" (UID: \"3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86b9db775-lkwv2" Jul 02 21:37:09.459251 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.459167 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/27a3595e-6df7-4999-8257-5c111b074330-original-pull-secret\") pod \"global-pull-secret-syncer-bg9p7\" (UID: \"27a3595e-6df7-4999-8257-5c111b074330\") " pod="kube-system/global-pull-secret-syncer-bg9p7" Jul 02 21:37:09.460281 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.460251 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:09.462527 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.462510 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 21:37:09.472886 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.472853 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/27a3595e-6df7-4999-8257-5c111b074330-original-pull-secret\") pod \"global-pull-secret-syncer-bg9p7\" (UID: \"27a3595e-6df7-4999-8257-5c111b074330\") " pod="kube-system/global-pull-secret-syncer-bg9p7" Jul 02 21:37:09.503616 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.503594 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b9cc8b67b-hxfmp" Jul 02 21:37:09.509906 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.509888 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86b9db775-lkwv2" Jul 02 21:37:09.559137 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.559109 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:09.560016 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.559995 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bc3fefa-dff5-4181-a239-c6573b8b4356-metrics-certs\") pod \"network-metrics-daemon-zp6sf\" (UID: \"0bc3fefa-dff5-4181-a239-c6573b8b4356\") " pod="openshift-multus/network-metrics-daemon-zp6sf" Jul 02 21:37:09.563020 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.562784 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 21:37:09.573268 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.573248 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bc3fefa-dff5-4181-a239-c6573b8b4356-metrics-certs\") pod \"network-metrics-daemon-zp6sf\" (UID: \"0bc3fefa-dff5-4181-a239-c6573b8b4356\") " pod="openshift-multus/network-metrics-daemon-zp6sf" Jul 02 21:37:09.625924 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.625895 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bg9p7" Jul 02 21:37:09.633134 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.633116 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dkhhs\"" Jul 02 21:37:09.641254 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.641223 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zp6sf" Jul 02 21:37:09.657214 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.657186 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:09.661157 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.661139 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnqz8\" (UniqueName: \"kubernetes.io/projected/298fd481-12c6-4869-a4d4-9b17c492706a-kube-api-access-cnqz8\") pod \"network-check-target-zkjd5\" (UID: \"298fd481-12c6-4869-a4d4-9b17c492706a\") " pod="openshift-network-diagnostics/network-check-target-zkjd5" Jul 02 21:37:09.663906 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.663886 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 21:37:09.675541 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.675517 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 21:37:09.685112 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.685089 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnqz8\" (UniqueName: \"kubernetes.io/projected/298fd481-12c6-4869-a4d4-9b17c492706a-kube-api-access-cnqz8\") pod \"network-check-target-zkjd5\" (UID: \"298fd481-12c6-4869-a4d4-9b17c492706a\") " pod="openshift-network-diagnostics/network-check-target-zkjd5" Jul 02 21:37:09.726004 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.725943 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:09.923503 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.923474 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ckjxv\"" Jul 02 21:37:09.931778 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:09.931753 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkjd5" Jul 02 21:37:10.554633 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:10.554603 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b9cc8b67b-hxfmp"] Jul 02 21:37:10.558858 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:37:10.558833 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda0ce982a_5708_426c_b332_2271f0d81c22.slice/crio-8f8ddfe7ed55cd87beee15b9b3656d5575dcd6a6e7854182d56d89a045a7bd92 WatchSource:0}: Error finding container 8f8ddfe7ed55cd87beee15b9b3656d5575dcd6a6e7854182d56d89a045a7bd92: Status 404 returned error can't find the container with id 8f8ddfe7ed55cd87beee15b9b3656d5575dcd6a6e7854182d56d89a045a7bd92 Jul 02 21:37:10.571792 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:10.571766 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zp6sf"] Jul 02 21:37:10.576227 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:37:10.576189 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bc3fefa_dff5_4181_a239_c6573b8b4356.slice/crio-83a07b4c04e84ed962bc05df1d9006bd791dd99a37148efb6c364ca2cda08e0b WatchSource:0}: Error finding container 83a07b4c04e84ed962bc05df1d9006bd791dd99a37148efb6c364ca2cda08e0b: Status 404 returned error can't find the container with id 83a07b4c04e84ed962bc05df1d9006bd791dd99a37148efb6c364ca2cda08e0b Jul 02 21:37:10.602007 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:10.601980 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86b9db775-lkwv2"] Jul 02 21:37:10.606748 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:37:10.606711 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3f0bfef3_8543_4ee0_b0f5_6824d0f14ef9.slice/crio-b5945fcb3d0f2e091dc2e3e6025bd54c832fa4d4acf06474b8f3f55bdbd7f041 WatchSource:0}: Error finding container b5945fcb3d0f2e091dc2e3e6025bd54c832fa4d4acf06474b8f3f55bdbd7f041: Status 404 returned error can't find the container with id b5945fcb3d0f2e091dc2e3e6025bd54c832fa4d4acf06474b8f3f55bdbd7f041 Jul 02 21:37:10.614442 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:10.614371 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bg9p7"] Jul 02 21:37:10.616489 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:37:10.616469 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27a3595e_6df7_4999_8257_5c111b074330.slice/crio-a5aa35c2481bc432f82f4ed801f4ad1d1ee54febac410556295b40a1470fcf2f WatchSource:0}: Error finding container a5aa35c2481bc432f82f4ed801f4ad1d1ee54febac410556295b40a1470fcf2f: Status 404 returned error can't find the container with id a5aa35c2481bc432f82f4ed801f4ad1d1ee54febac410556295b40a1470fcf2f Jul 02 21:37:10.698122 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:10.698057 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:10.771688 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:10.771661 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zkjd5"] Jul 02 21:37:10.774195 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:37:10.774172 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod298fd481_12c6_4869_a4d4_9b17c492706a.slice/crio-d5c0003e8c71ee7b91c97e680677a5b2e1f2c1643b442545ffe27b528fe396ba WatchSource:0}: Error finding container d5c0003e8c71ee7b91c97e680677a5b2e1f2c1643b442545ffe27b528fe396ba: Status 404 returned error can't find the container with id d5c0003e8c71ee7b91c97e680677a5b2e1f2c1643b442545ffe27b528fe396ba Jul 02 21:37:11.089512 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:11.089478 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:11.118399 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:11.118362 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zp6sf" event={"ID":"0bc3fefa-dff5-4181-a239-c6573b8b4356","Type":"ContainerStarted","Data":"83a07b4c04e84ed962bc05df1d9006bd791dd99a37148efb6c364ca2cda08e0b"} Jul 02 21:37:11.119964 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:11.119904 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zkjd5" event={"ID":"298fd481-12c6-4869-a4d4-9b17c492706a","Type":"ContainerStarted","Data":"d5c0003e8c71ee7b91c97e680677a5b2e1f2c1643b442545ffe27b528fe396ba"} Jul 02 21:37:11.122493 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:11.122469 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lzqbp" event={"ID":"245f2091-5cf4-4185-a0fb-629ec98a0be0","Type":"ContainerStarted","Data":"75ca29588af9b9f8f4d48c144bed14a025d0a7d3d36b6ef4bf1605d9720963bd"} Jul 02 21:37:11.124231 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:11.124207 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b9cc8b67b-hxfmp" event={"ID":"a0ce982a-5708-426c-b332-2271f0d81c22","Type":"ContainerStarted","Data":"8f8ddfe7ed55cd87beee15b9b3656d5575dcd6a6e7854182d56d89a045a7bd92"} Jul 02 21:37:11.125765 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:11.125737 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86b9db775-lkwv2" event={"ID":"3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9","Type":"ContainerStarted","Data":"b5945fcb3d0f2e091dc2e3e6025bd54c832fa4d4acf06474b8f3f55bdbd7f041"} Jul 02 21:37:11.127013 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:11.126967 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bg9p7" event={"ID":"27a3595e-6df7-4999-8257-5c111b074330","Type":"ContainerStarted","Data":"a5aa35c2481bc432f82f4ed801f4ad1d1ee54febac410556295b40a1470fcf2f"} Jul 02 21:37:11.144616 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:11.144584 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:11.878199 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:11.878154 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/487a1265-7879-4ca9-a747-40bbdba201c4-metrics-tls\") pod \"dns-default-m7c4n\" (UID: \"487a1265-7879-4ca9-a747-40bbdba201c4\") " pod="openshift-dns/dns-default-m7c4n" Jul 02 21:37:11.881614 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:11.881560 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/487a1265-7879-4ca9-a747-40bbdba201c4-metrics-tls\") pod \"dns-default-m7c4n\" (UID: \"487a1265-7879-4ca9-a747-40bbdba201c4\") " pod="openshift-dns/dns-default-m7c4n" Jul 02 21:37:12.177334 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:12.177254 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bv4sq\"" Jul 02 21:37:12.185354 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:12.185133 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-m7c4n" Jul 02 21:37:12.411169 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:12.411085 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lzqbp" podStartSLOduration=18.718377996 podStartE2EDuration="25.411069493s" totalImagesPullingTime="6.692691497s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 21:36:47 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:37:11.141352274 +0000 UTC m=+67.860701816" watchObservedRunningTime="2026-07-02 21:37:12.411069493 +0000 UTC m=+69.130419034" Jul 02 21:37:12.412144 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:12.411765 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-m7c4n"] Jul 02 21:37:12.415817 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:37:12.415786 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod487a1265_7879_4ca9_a747_40bbdba201c4.slice/crio-48a93134b4c5b071265d758f55b2c47164c37406350980092ce459dd8731f0c9 WatchSource:0}: Error finding container 48a93134b4c5b071265d758f55b2c47164c37406350980092ce459dd8731f0c9: Status 404 returned error can't find the container with id 48a93134b4c5b071265d758f55b2c47164c37406350980092ce459dd8731f0c9 Jul 02 21:37:13.139360 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:13.139319 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m7c4n" event={"ID":"487a1265-7879-4ca9-a747-40bbdba201c4","Type":"ContainerStarted","Data":"48a93134b4c5b071265d758f55b2c47164c37406350980092ce459dd8731f0c9"} Jul 02 21:37:13.141981 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:13.141943 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zp6sf" event={"ID":"0bc3fefa-dff5-4181-a239-c6573b8b4356","Type":"ContainerStarted","Data":"ef7484daa529d6a47c18cec16ed40fae0a3cdec185f45c3f3fcd21921c3a8d68"} Jul 02 21:37:13.142119 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:13.142001 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zp6sf" event={"ID":"0bc3fefa-dff5-4181-a239-c6573b8b4356","Type":"ContainerStarted","Data":"4b282f375598da573e32ccb3ce1473a749c070229944e913ce7558cc02e9e318"} Jul 02 21:37:13.158773 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:13.157715 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-zp6sf" podStartSLOduration=68.585078627 podStartE2EDuration="1m10.157696355s" totalImagesPullingTime="1.572617728s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:36:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:37:13.157271333 +0000 UTC m=+69.876620875" watchObservedRunningTime="2026-07-02 21:37:13.157696355 +0000 UTC m=+69.877045897" Jul 02 21:37:13.244497 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:13.244464 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:13.343846 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:13.343754 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:14.712636 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:14.712607 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:14.808591 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:14.808541 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:14.872346 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:14.872307 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:14.965214 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:14.965129 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:15.364341 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:15.364306 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:15.460602 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:15.460565 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:15.562032 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:15.562004 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:15.663828 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:15.663739 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:15.762308 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:15.762274 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:16.031174 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:16.031136 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:16.082525 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:16.082492 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:16.239234 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:16.239190 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:16.289822 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:16.289739 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:16.439033 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:16.439000 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:16.494247 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:16.494214 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:16.629558 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:16.629478 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:16.682157 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:16.682127 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:17.042302 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:17.042268 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:17.176924 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:17.176890 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:17.277393 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:17.277362 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:18.857634 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:18.857581 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:18.958287 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:18.958257 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:19.171175 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:19.171088 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b9cc8b67b-hxfmp" event={"ID":"a0ce982a-5708-426c-b332-2271f0d81c22","Type":"ContainerStarted","Data":"299723642e6d38b8a7445df56c586b194b00986d76f53a5c1e11129f65257a72"} Jul 02 21:37:19.173071 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:19.173039 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m7c4n" event={"ID":"487a1265-7879-4ca9-a747-40bbdba201c4","Type":"ContainerStarted","Data":"91fc865c7db562480a19b7ad5466ce50f9f1211dc3f442955d2bbcffb1487c53"} Jul 02 21:37:19.173224 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:19.173091 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m7c4n" event={"ID":"487a1265-7879-4ca9-a747-40bbdba201c4","Type":"ContainerStarted","Data":"aab79136efc7f9a3aa4dfebfc276d0b8a1af7e068020eb32929d0ee848892ee0"} Jul 02 21:37:19.173224 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:19.173135 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-m7c4n" Jul 02 21:37:19.175192 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:19.175168 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86b9db775-lkwv2" event={"ID":"3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9","Type":"ContainerStarted","Data":"324ab96a75ce480159f68a62de5f27065bdf7e369e2639386ad90cbae536934b"} Jul 02 21:37:19.175300 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:19.175203 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86b9db775-lkwv2" event={"ID":"3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9","Type":"ContainerStarted","Data":"a950da65af0dd7e46dc9fc9cb09dd43c151b5bc5136704aedf064ccbd737ccea"} Jul 02 21:37:19.175300 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:19.175223 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86b9db775-lkwv2" event={"ID":"3f0bfef3-8543-4ee0-b0f5-6824d0f14ef9","Type":"ContainerStarted","Data":"e0ac759c4f31c863bc42b5eb923925afc4228d337ead50db7c0cc16aa46b613d"} Jul 02 21:37:19.176892 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:19.176840 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bg9p7" event={"ID":"27a3595e-6df7-4999-8257-5c111b074330","Type":"ContainerStarted","Data":"55b72cabafd41aaedef9ceb5112b084e11eb1b857d5a5e5945c7ea759801056c"} Jul 02 21:37:19.181262 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:19.181231 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zkjd5" event={"ID":"298fd481-12c6-4869-a4d4-9b17c492706a","Type":"ContainerStarted","Data":"eacac2e168d57129d945ff8a92225dca8a5ae5d79dd87ee5ebcb6495b92dd139"} Jul 02 21:37:19.181356 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:19.181309 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zkjd5" Jul 02 21:37:19.186985 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:19.186947 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b9cc8b67b-hxfmp" podStartSLOduration=2.086983691 podStartE2EDuration="10.186935659s" totalImagesPullingTime="8.099951968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:37:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:37:19.185402648 +0000 UTC m=+75.904752190" watchObservedRunningTime="2026-07-02 21:37:19.186935659 +0000 UTC m=+75.906285200" Jul 02 21:37:19.199502 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:19.199462 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-bg9p7" podStartSLOduration=68.138476671 podStartE2EDuration="1m16.199451285s" totalImagesPullingTime="8.060974614s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 21:36:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:37:19.198956552 +0000 UTC m=+75.918306090" watchObservedRunningTime="2026-07-02 21:37:19.199451285 +0000 UTC m=+75.918800803" Jul 02 21:37:19.214972 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:19.214935 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-m7c4n" podStartSLOduration=32.970477643 podStartE2EDuration="39.214922852s" totalImagesPullingTime="6.244445209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:36:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:37:19.214511862 +0000 UTC m=+75.933861403" watchObservedRunningTime="2026-07-02 21:37:19.214922852 +0000 UTC m=+75.934272620" Jul 02 21:37:19.230045 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:19.230003 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-zkjd5" podStartSLOduration=68.334055682 podStartE2EDuration="1m16.229992145s" totalImagesPullingTime="7.895936463s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:36:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:37:19.229484679 +0000 UTC m=+75.948834215" watchObservedRunningTime="2026-07-02 21:37:19.229992145 +0000 UTC m=+75.949341686" Jul 02 21:37:19.909916 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:19.909862 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:19.976607 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:19.976578 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:20.272084 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:20.272053 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:22.528994 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:22.528955 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:25.028766 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.028631 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:25.263432 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.263403 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:25.323145 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.323053 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86b9db775-lkwv2" podStartSLOduration=8.252060122 podStartE2EDuration="16.323036531s" totalImagesPullingTime="8.070976409s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:37:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:37:19.248756355 +0000 UTC m=+75.968105896" watchObservedRunningTime="2026-07-02 21:37:25.323036531 +0000 UTC m=+82.042386072" Jul 02 21:37:25.323277 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.323195 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-gwshn"] Jul 02 21:37:25.325540 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.325525 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gwshn" Jul 02 21:37:25.328385 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.328364 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 02 21:37:25.328765 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.328748 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 02 21:37:25.328838 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.328797 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-p4xt9\"" Jul 02 21:37:25.329734 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.329712 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 02 21:37:25.329848 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.329775 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 02 21:37:25.329848 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.329808 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 02 21:37:25.329848 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.329822 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 02 21:37:25.355263 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.355236 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:25.383373 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.383347 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d4eab772-b0dd-4373-86f5-1ea1b897b14f-node-exporter-accelerators-collector-config\") pod \"node-exporter-gwshn\" (UID: \"d4eab772-b0dd-4373-86f5-1ea1b897b14f\") " pod="openshift-monitoring/node-exporter-gwshn" Jul 02 21:37:25.383476 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.383393 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d4eab772-b0dd-4373-86f5-1ea1b897b14f-root\") pod \"node-exporter-gwshn\" (UID: \"d4eab772-b0dd-4373-86f5-1ea1b897b14f\") " pod="openshift-monitoring/node-exporter-gwshn" Jul 02 21:37:25.383476 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.383426 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d4eab772-b0dd-4373-86f5-1ea1b897b14f-sys\") pod \"node-exporter-gwshn\" (UID: \"d4eab772-b0dd-4373-86f5-1ea1b897b14f\") " pod="openshift-monitoring/node-exporter-gwshn" Jul 02 21:37:25.383476 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.383455 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7t7lx\" (UniqueName: \"kubernetes.io/projected/d4eab772-b0dd-4373-86f5-1ea1b897b14f-kube-api-access-7t7lx\") pod \"node-exporter-gwshn\" (UID: \"d4eab772-b0dd-4373-86f5-1ea1b897b14f\") " pod="openshift-monitoring/node-exporter-gwshn" Jul 02 21:37:25.383579 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.383495 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d4eab772-b0dd-4373-86f5-1ea1b897b14f-node-exporter-textfile\") pod \"node-exporter-gwshn\" (UID: \"d4eab772-b0dd-4373-86f5-1ea1b897b14f\") " pod="openshift-monitoring/node-exporter-gwshn" Jul 02 21:37:25.383579 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.383537 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d4eab772-b0dd-4373-86f5-1ea1b897b14f-node-exporter-tls\") pod \"node-exporter-gwshn\" (UID: \"d4eab772-b0dd-4373-86f5-1ea1b897b14f\") " pod="openshift-monitoring/node-exporter-gwshn" Jul 02 21:37:25.383579 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.383569 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d4eab772-b0dd-4373-86f5-1ea1b897b14f-node-exporter-wtmp\") pod \"node-exporter-gwshn\" (UID: \"d4eab772-b0dd-4373-86f5-1ea1b897b14f\") " pod="openshift-monitoring/node-exporter-gwshn" Jul 02 21:37:25.383692 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.383589 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d4eab772-b0dd-4373-86f5-1ea1b897b14f-metrics-client-ca\") pod \"node-exporter-gwshn\" (UID: \"d4eab772-b0dd-4373-86f5-1ea1b897b14f\") " pod="openshift-monitoring/node-exporter-gwshn" Jul 02 21:37:25.383692 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.383651 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d4eab772-b0dd-4373-86f5-1ea1b897b14f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gwshn\" (UID: \"d4eab772-b0dd-4373-86f5-1ea1b897b14f\") " pod="openshift-monitoring/node-exporter-gwshn" Jul 02 21:37:25.421121 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.421095 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:25.484940 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.484915 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d4eab772-b0dd-4373-86f5-1ea1b897b14f-node-exporter-textfile\") pod \"node-exporter-gwshn\" (UID: \"d4eab772-b0dd-4373-86f5-1ea1b897b14f\") " pod="openshift-monitoring/node-exporter-gwshn" Jul 02 21:37:25.485057 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.484946 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d4eab772-b0dd-4373-86f5-1ea1b897b14f-node-exporter-tls\") pod \"node-exporter-gwshn\" (UID: \"d4eab772-b0dd-4373-86f5-1ea1b897b14f\") " pod="openshift-monitoring/node-exporter-gwshn" Jul 02 21:37:25.485057 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.484970 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d4eab772-b0dd-4373-86f5-1ea1b897b14f-node-exporter-wtmp\") pod \"node-exporter-gwshn\" (UID: \"d4eab772-b0dd-4373-86f5-1ea1b897b14f\") " pod="openshift-monitoring/node-exporter-gwshn" Jul 02 21:37:25.485057 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.484987 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d4eab772-b0dd-4373-86f5-1ea1b897b14f-metrics-client-ca\") pod \"node-exporter-gwshn\" (UID: \"d4eab772-b0dd-4373-86f5-1ea1b897b14f\") " pod="openshift-monitoring/node-exporter-gwshn" Jul 02 21:37:25.485057 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.485007 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d4eab772-b0dd-4373-86f5-1ea1b897b14f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gwshn\" (UID: \"d4eab772-b0dd-4373-86f5-1ea1b897b14f\") " pod="openshift-monitoring/node-exporter-gwshn" Jul 02 21:37:25.485057 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.485026 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d4eab772-b0dd-4373-86f5-1ea1b897b14f-node-exporter-accelerators-collector-config\") pod \"node-exporter-gwshn\" (UID: \"d4eab772-b0dd-4373-86f5-1ea1b897b14f\") " pod="openshift-monitoring/node-exporter-gwshn" Jul 02 21:37:25.485297 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.485070 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d4eab772-b0dd-4373-86f5-1ea1b897b14f-root\") pod \"node-exporter-gwshn\" (UID: \"d4eab772-b0dd-4373-86f5-1ea1b897b14f\") " pod="openshift-monitoring/node-exporter-gwshn" Jul 02 21:37:25.485297 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.485097 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d4eab772-b0dd-4373-86f5-1ea1b897b14f-sys\") pod \"node-exporter-gwshn\" (UID: \"d4eab772-b0dd-4373-86f5-1ea1b897b14f\") " pod="openshift-monitoring/node-exporter-gwshn" Jul 02 21:37:25.485297 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:37:25.485105 2556 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 02 21:37:25.485297 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.485116 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7t7lx\" (UniqueName: \"kubernetes.io/projected/d4eab772-b0dd-4373-86f5-1ea1b897b14f-kube-api-access-7t7lx\") pod \"node-exporter-gwshn\" (UID: \"d4eab772-b0dd-4373-86f5-1ea1b897b14f\") " pod="openshift-monitoring/node-exporter-gwshn" Jul 02 21:37:25.485297 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.485147 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d4eab772-b0dd-4373-86f5-1ea1b897b14f-node-exporter-wtmp\") pod \"node-exporter-gwshn\" (UID: \"d4eab772-b0dd-4373-86f5-1ea1b897b14f\") " pod="openshift-monitoring/node-exporter-gwshn" Jul 02 21:37:25.485297 ip-10-0-140-146 kubenswrapper[2556]: E0702 21:37:25.485180 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4eab772-b0dd-4373-86f5-1ea1b897b14f-node-exporter-tls podName:d4eab772-b0dd-4373-86f5-1ea1b897b14f nodeName:}" failed. No retries permitted until 2026-07-02 21:37:25.985158943 +0000 UTC m=+82.704508490 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/d4eab772-b0dd-4373-86f5-1ea1b897b14f-node-exporter-tls") pod "node-exporter-gwshn" (UID: "d4eab772-b0dd-4373-86f5-1ea1b897b14f") : secret "node-exporter-tls" not found Jul 02 21:37:25.485297 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.485181 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d4eab772-b0dd-4373-86f5-1ea1b897b14f-sys\") pod \"node-exporter-gwshn\" (UID: \"d4eab772-b0dd-4373-86f5-1ea1b897b14f\") " pod="openshift-monitoring/node-exporter-gwshn" Jul 02 21:37:25.485297 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.485249 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d4eab772-b0dd-4373-86f5-1ea1b897b14f-root\") pod \"node-exporter-gwshn\" (UID: \"d4eab772-b0dd-4373-86f5-1ea1b897b14f\") " pod="openshift-monitoring/node-exporter-gwshn" Jul 02 21:37:25.485621 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.485325 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d4eab772-b0dd-4373-86f5-1ea1b897b14f-node-exporter-textfile\") pod \"node-exporter-gwshn\" (UID: \"d4eab772-b0dd-4373-86f5-1ea1b897b14f\") " pod="openshift-monitoring/node-exporter-gwshn" Jul 02 21:37:25.485621 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.485558 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d4eab772-b0dd-4373-86f5-1ea1b897b14f-metrics-client-ca\") pod \"node-exporter-gwshn\" (UID: \"d4eab772-b0dd-4373-86f5-1ea1b897b14f\") " pod="openshift-monitoring/node-exporter-gwshn" Jul 02 21:37:25.485730 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.485640 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d4eab772-b0dd-4373-86f5-1ea1b897b14f-node-exporter-accelerators-collector-config\") pod \"node-exporter-gwshn\" (UID: \"d4eab772-b0dd-4373-86f5-1ea1b897b14f\") " pod="openshift-monitoring/node-exporter-gwshn" Jul 02 21:37:25.487376 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.487359 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d4eab772-b0dd-4373-86f5-1ea1b897b14f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gwshn\" (UID: \"d4eab772-b0dd-4373-86f5-1ea1b897b14f\") " pod="openshift-monitoring/node-exporter-gwshn" Jul 02 21:37:25.499073 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.499054 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7t7lx\" (UniqueName: \"kubernetes.io/projected/d4eab772-b0dd-4373-86f5-1ea1b897b14f-kube-api-access-7t7lx\") pod \"node-exporter-gwshn\" (UID: \"d4eab772-b0dd-4373-86f5-1ea1b897b14f\") " pod="openshift-monitoring/node-exporter-gwshn" Jul 02 21:37:25.561334 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.561304 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:25.756975 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.756949 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:25.856837 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.856806 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:25.926130 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.926098 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:25.987525 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.987495 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d4eab772-b0dd-4373-86f5-1ea1b897b14f-node-exporter-tls\") pod \"node-exporter-gwshn\" (UID: \"d4eab772-b0dd-4373-86f5-1ea1b897b14f\") " pod="openshift-monitoring/node-exporter-gwshn" Jul 02 21:37:25.989915 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:25.989896 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d4eab772-b0dd-4373-86f5-1ea1b897b14f-node-exporter-tls\") pod \"node-exporter-gwshn\" (UID: \"d4eab772-b0dd-4373-86f5-1ea1b897b14f\") " pod="openshift-monitoring/node-exporter-gwshn" Jul 02 21:37:26.178075 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:26.177993 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:26.234767 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:26.234737 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gwshn" Jul 02 21:37:26.244806 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:37:26.244780 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4eab772_b0dd_4373_86f5_1ea1b897b14f.slice/crio-86da2e1450970a1c10e222c39576b23a3b2fba8a09f01f677779c5811e9dc6d2 WatchSource:0}: Error finding container 86da2e1450970a1c10e222c39576b23a3b2fba8a09f01f677779c5811e9dc6d2: Status 404 returned error can't find the container with id 86da2e1450970a1c10e222c39576b23a3b2fba8a09f01f677779c5811e9dc6d2 Jul 02 21:37:26.278200 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:26.278171 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:26.340495 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:26.340464 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:26.800592 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:26.800558 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:27.187228 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:27.187204 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:27.208351 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:27.208326 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gwshn" event={"ID":"d4eab772-b0dd-4373-86f5-1ea1b897b14f","Type":"ContainerStarted","Data":"86da2e1450970a1c10e222c39576b23a3b2fba8a09f01f677779c5811e9dc6d2"} Jul 02 21:37:27.255268 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:27.255238 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:28.214426 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:28.214391 2556 generic.go:356] "Generic (PLEG): container finished" podID="d4eab772-b0dd-4373-86f5-1ea1b897b14f" containerID="b7e5fa29061684960293f1b90ce27d3952f148d36f6edb5b627ce1c0cf58cbcd" exitCode=0 Jul 02 21:37:28.214858 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:28.214481 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gwshn" event={"ID":"d4eab772-b0dd-4373-86f5-1ea1b897b14f","Type":"ContainerDied","Data":"b7e5fa29061684960293f1b90ce27d3952f148d36f6edb5b627ce1c0cf58cbcd"} Jul 02 21:37:28.261368 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:28.261343 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:28.362181 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:28.362158 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:28.543767 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:28.543736 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:28.645646 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:28.645618 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:29.186824 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:29.186795 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-m7c4n" Jul 02 21:37:29.221029 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:29.221000 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gwshn" event={"ID":"d4eab772-b0dd-4373-86f5-1ea1b897b14f","Type":"ContainerStarted","Data":"28f7441396db404446f1b0c1ba90e9b5a42855dd3d1e12364d92538406366c07"} Jul 02 21:37:29.221459 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:29.221049 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gwshn" event={"ID":"d4eab772-b0dd-4373-86f5-1ea1b897b14f","Type":"ContainerStarted","Data":"62fc114ef73b38e6e6c27c7ce030d18ee81ceebf0f86b74166a190c3694ccee8"} Jul 02 21:37:29.239999 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:29.239941 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-gwshn" podStartSLOduration=3.24333363 podStartE2EDuration="4.239920756s" totalImagesPullingTime="925.553919ms" totalInitContainerRuntime="71.033207ms" isStatefulPod=true podCreationTimestamp="2026-07-02 21:37:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:37:29.238344055 +0000 UTC m=+85.957693596" watchObservedRunningTime="2026-07-02 21:37:29.239920756 +0000 UTC m=+85.959270301" Jul 02 21:37:30.046665 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:30.046629 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:30.121843 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:30.121808 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:30.178997 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:30.178971 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:30.302410 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:30.302328 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:31.535883 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:31.535850 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:31.604522 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:31.604488 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:31.669041 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:31.669016 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:32.254474 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:32.254439 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:32.453467 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:32.453437 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:32.510591 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:32.510564 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:34.129015 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:34.128982 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:34.229371 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:34.229346 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:37.472095 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:37.472065 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:37.764892 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:37.764855 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:37.816306 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:37.816280 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:39.111144 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:39.111111 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:39.166964 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:39.166939 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:43.133131 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:43.133095 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:43.237264 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:43.237231 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:46.469468 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:46.469430 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:46.521656 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:46.521626 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:46.635292 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:46.635238 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:46.691475 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:46.691452 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:46.842386 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:46.842355 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:46.943228 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:46.943196 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:49.468751 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:49.468717 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:49.569665 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:49.569640 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:50.187599 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:50.187572 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-zkjd5" Jul 02 21:37:50.671759 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:50.671731 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:37:50.773388 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:37:50.773356 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:38:01.633157 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:38:01.633121 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:38:01.700029 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:38:01.699996 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:38:11.991068 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:38:11.991036 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:38:41.475323 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:38:41.475290 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:38:41.572567 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:38:41.572537 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:38:53.421551 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:38:53.421524 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:38:53.482611 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:38:53.482579 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:38:55.459584 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:38:55.459509 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:38:55.515726 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:38:55.515695 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:39:01.643279 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:39:01.643248 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:39:01.742590 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:39:01.742562 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:39:06.667681 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:39:06.667645 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:39:06.769537 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:39:06.769506 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:39:06.824929 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:39:06.824900 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:39:22.045130 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:39:22.045098 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:39:35.779718 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:39:35.779683 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:39:35.935688 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:39:35.935660 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:39:36.033604 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:39:36.033525 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:39:42.960013 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:39:42.959975 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:39:43.016258 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:39:43.016225 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:39:58.379909 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:39:58.379857 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:39:58.577555 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:39:58.577516 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:40:02.848312 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:02.848274 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:40:02.988555 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:02.988522 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:40:04.318372 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:04.318344 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:40:04.490313 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:04.490280 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:40:04.773763 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:04.773733 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:40:04.933949 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:04.933915 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:40:07.045716 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:07.045683 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:40:07.371817 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:07.371734 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:40:08.561974 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:08.561942 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:40:08.616861 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:08.616837 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:40:14.777600 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:14.777566 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:40:14.877327 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:14.877296 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:40:14.976596 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:14.976568 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:40:15.236301 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:15.236229 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rgc77"] Jul 02 21:40:15.238553 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:15.238537 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-rgc77" Jul 02 21:40:15.241007 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:15.240986 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 02 21:40:15.241007 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:15.241001 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 02 21:40:15.242113 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:15.242097 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-cnrc9\"" Jul 02 21:40:15.249415 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:15.249395 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rgc77"] Jul 02 21:40:15.316531 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:15.316510 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwzf9\" (UniqueName: \"kubernetes.io/projected/53323ef1-e3d5-4151-85d0-27cd6079e831-kube-api-access-hwzf9\") pod \"cert-manager-55b4d954d5-rgc77\" (UID: \"53323ef1-e3d5-4151-85d0-27cd6079e831\") " pod="cert-manager/cert-manager-55b4d954d5-rgc77" Jul 02 21:40:15.316654 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:15.316548 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/53323ef1-e3d5-4151-85d0-27cd6079e831-bound-sa-token\") pod \"cert-manager-55b4d954d5-rgc77\" (UID: \"53323ef1-e3d5-4151-85d0-27cd6079e831\") " pod="cert-manager/cert-manager-55b4d954d5-rgc77" Jul 02 21:40:15.417435 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:15.417399 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwzf9\" (UniqueName: \"kubernetes.io/projected/53323ef1-e3d5-4151-85d0-27cd6079e831-kube-api-access-hwzf9\") pod \"cert-manager-55b4d954d5-rgc77\" (UID: \"53323ef1-e3d5-4151-85d0-27cd6079e831\") " pod="cert-manager/cert-manager-55b4d954d5-rgc77" Jul 02 21:40:15.417560 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:15.417449 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/53323ef1-e3d5-4151-85d0-27cd6079e831-bound-sa-token\") pod \"cert-manager-55b4d954d5-rgc77\" (UID: \"53323ef1-e3d5-4151-85d0-27cd6079e831\") " pod="cert-manager/cert-manager-55b4d954d5-rgc77" Jul 02 21:40:15.425826 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:15.425796 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/53323ef1-e3d5-4151-85d0-27cd6079e831-bound-sa-token\") pod \"cert-manager-55b4d954d5-rgc77\" (UID: \"53323ef1-e3d5-4151-85d0-27cd6079e831\") " pod="cert-manager/cert-manager-55b4d954d5-rgc77" Jul 02 21:40:15.425949 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:15.425932 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwzf9\" (UniqueName: \"kubernetes.io/projected/53323ef1-e3d5-4151-85d0-27cd6079e831-kube-api-access-hwzf9\") pod \"cert-manager-55b4d954d5-rgc77\" (UID: \"53323ef1-e3d5-4151-85d0-27cd6079e831\") " pod="cert-manager/cert-manager-55b4d954d5-rgc77" Jul 02 21:40:15.547437 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:15.547417 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-rgc77" Jul 02 21:40:15.670064 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:15.670041 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rgc77"] Jul 02 21:40:15.672545 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:40:15.672519 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod53323ef1_e3d5_4151_85d0_27cd6079e831.slice/crio-5108c62c99c092843c5374ca47911f77e090791c558b02e84642b343148e998e WatchSource:0}: Error finding container 5108c62c99c092843c5374ca47911f77e090791c558b02e84642b343148e998e: Status 404 returned error can't find the container with id 5108c62c99c092843c5374ca47911f77e090791c558b02e84642b343148e998e Jul 02 21:40:15.744841 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:15.744810 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-rgc77" event={"ID":"53323ef1-e3d5-4151-85d0-27cd6079e831","Type":"ContainerStarted","Data":"5108c62c99c092843c5374ca47911f77e090791c558b02e84642b343148e998e"} Jul 02 21:40:19.034714 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:19.034673 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:40:19.760192 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:19.760162 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-rgc77" event={"ID":"53323ef1-e3d5-4151-85d0-27cd6079e831","Type":"ContainerStarted","Data":"43e38dc67d942e250bccd284f8c6427eb65ccfd99ec8d54f1529133ae74c26df"} Jul 02 21:40:19.779650 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:19.779597 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-rgc77" podStartSLOduration=1.622235525 podStartE2EDuration="4.779580708s" totalImagesPullingTime="3.157345183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:40:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:40:19.778321181 +0000 UTC m=+256.497670721" watchObservedRunningTime="2026-07-02 21:40:19.779580708 +0000 UTC m=+256.498930236" Jul 02 21:40:23.366765 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:23.366731 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:40:23.461889 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:23.461836 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:40:29.902131 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:29.902053 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:40:30.003372 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:30.003338 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:40:31.970700 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:31.970664 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:40:38.208827 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:38.208794 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:40:38.263542 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:38.263515 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-8677d7b8dc-xtgj6"] Jul 02 21:40:38.268418 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:38.268401 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8677d7b8dc-xtgj6" Jul 02 21:40:38.272143 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:38.272119 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 02 21:40:38.272285 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:38.272199 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 02 21:40:38.272285 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:38.272216 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 02 21:40:38.272285 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:38.272238 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 02 21:40:38.272285 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:38.272199 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 02 21:40:38.272505 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:38.272338 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-kv67n\"" Jul 02 21:40:38.281484 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:38.281462 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8677d7b8dc-xtgj6"] Jul 02 21:40:38.312954 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:38.312923 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:40:38.377730 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:38.377702 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d79c0f71-febe-42f6-94eb-728225cd99fe-cert\") pod \"lws-controller-manager-8677d7b8dc-xtgj6\" (UID: \"d79c0f71-febe-42f6-94eb-728225cd99fe\") " pod="openshift-lws-operator/lws-controller-manager-8677d7b8dc-xtgj6" Jul 02 21:40:38.377909 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:38.377735 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d79c0f71-febe-42f6-94eb-728225cd99fe-manager-config\") pod \"lws-controller-manager-8677d7b8dc-xtgj6\" (UID: \"d79c0f71-febe-42f6-94eb-728225cd99fe\") " pod="openshift-lws-operator/lws-controller-manager-8677d7b8dc-xtgj6" Jul 02 21:40:38.377909 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:38.377762 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d79c0f71-febe-42f6-94eb-728225cd99fe-metrics-cert\") pod \"lws-controller-manager-8677d7b8dc-xtgj6\" (UID: \"d79c0f71-febe-42f6-94eb-728225cd99fe\") " pod="openshift-lws-operator/lws-controller-manager-8677d7b8dc-xtgj6" Jul 02 21:40:38.377909 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:38.377854 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-699dx\" (UniqueName: \"kubernetes.io/projected/d79c0f71-febe-42f6-94eb-728225cd99fe-kube-api-access-699dx\") pod \"lws-controller-manager-8677d7b8dc-xtgj6\" (UID: \"d79c0f71-febe-42f6-94eb-728225cd99fe\") " pod="openshift-lws-operator/lws-controller-manager-8677d7b8dc-xtgj6" Jul 02 21:40:38.478267 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:38.478194 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d79c0f71-febe-42f6-94eb-728225cd99fe-manager-config\") pod \"lws-controller-manager-8677d7b8dc-xtgj6\" (UID: \"d79c0f71-febe-42f6-94eb-728225cd99fe\") " pod="openshift-lws-operator/lws-controller-manager-8677d7b8dc-xtgj6" Jul 02 21:40:38.478267 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:38.478234 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d79c0f71-febe-42f6-94eb-728225cd99fe-metrics-cert\") pod \"lws-controller-manager-8677d7b8dc-xtgj6\" (UID: \"d79c0f71-febe-42f6-94eb-728225cd99fe\") " pod="openshift-lws-operator/lws-controller-manager-8677d7b8dc-xtgj6" Jul 02 21:40:38.478430 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:38.478266 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-699dx\" (UniqueName: \"kubernetes.io/projected/d79c0f71-febe-42f6-94eb-728225cd99fe-kube-api-access-699dx\") pod \"lws-controller-manager-8677d7b8dc-xtgj6\" (UID: \"d79c0f71-febe-42f6-94eb-728225cd99fe\") " pod="openshift-lws-operator/lws-controller-manager-8677d7b8dc-xtgj6" Jul 02 21:40:38.478430 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:38.478311 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d79c0f71-febe-42f6-94eb-728225cd99fe-cert\") pod \"lws-controller-manager-8677d7b8dc-xtgj6\" (UID: \"d79c0f71-febe-42f6-94eb-728225cd99fe\") " pod="openshift-lws-operator/lws-controller-manager-8677d7b8dc-xtgj6" Jul 02 21:40:38.478929 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:38.478907 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d79c0f71-febe-42f6-94eb-728225cd99fe-manager-config\") pod \"lws-controller-manager-8677d7b8dc-xtgj6\" (UID: \"d79c0f71-febe-42f6-94eb-728225cd99fe\") " pod="openshift-lws-operator/lws-controller-manager-8677d7b8dc-xtgj6" Jul 02 21:40:38.480916 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:38.480895 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d79c0f71-febe-42f6-94eb-728225cd99fe-metrics-cert\") pod \"lws-controller-manager-8677d7b8dc-xtgj6\" (UID: \"d79c0f71-febe-42f6-94eb-728225cd99fe\") " pod="openshift-lws-operator/lws-controller-manager-8677d7b8dc-xtgj6" Jul 02 21:40:38.481009 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:38.480899 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d79c0f71-febe-42f6-94eb-728225cd99fe-cert\") pod \"lws-controller-manager-8677d7b8dc-xtgj6\" (UID: \"d79c0f71-febe-42f6-94eb-728225cd99fe\") " pod="openshift-lws-operator/lws-controller-manager-8677d7b8dc-xtgj6" Jul 02 21:40:38.489505 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:38.489485 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-699dx\" (UniqueName: \"kubernetes.io/projected/d79c0f71-febe-42f6-94eb-728225cd99fe-kube-api-access-699dx\") pod \"lws-controller-manager-8677d7b8dc-xtgj6\" (UID: \"d79c0f71-febe-42f6-94eb-728225cd99fe\") " pod="openshift-lws-operator/lws-controller-manager-8677d7b8dc-xtgj6" Jul 02 21:40:38.578097 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:38.578054 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8677d7b8dc-xtgj6" Jul 02 21:40:38.702276 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:38.702238 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8677d7b8dc-xtgj6"] Jul 02 21:40:38.705586 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:40:38.705562 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd79c0f71_febe_42f6_94eb_728225cd99fe.slice/crio-e4b28a9eb0686bfecb022176428c1851ca56eb65babbdafa39d25b1589ed5011 WatchSource:0}: Error finding container e4b28a9eb0686bfecb022176428c1851ca56eb65babbdafa39d25b1589ed5011: Status 404 returned error can't find the container with id e4b28a9eb0686bfecb022176428c1851ca56eb65babbdafa39d25b1589ed5011 Jul 02 21:40:38.822639 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:38.822604 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8677d7b8dc-xtgj6" event={"ID":"d79c0f71-febe-42f6-94eb-728225cd99fe","Type":"ContainerStarted","Data":"e4b28a9eb0686bfecb022176428c1851ca56eb65babbdafa39d25b1589ed5011"} Jul 02 21:40:41.833775 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:41.833742 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8677d7b8dc-xtgj6" event={"ID":"d79c0f71-febe-42f6-94eb-728225cd99fe","Type":"ContainerStarted","Data":"15f3884dc572dd49a5729496ca60812699955562e7fb6c897a133e0149179722"} Jul 02 21:40:41.834160 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:41.833896 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-8677d7b8dc-xtgj6" Jul 02 21:40:41.853498 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:41.853396 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-8677d7b8dc-xtgj6" podStartSLOduration=0.824171662 podStartE2EDuration="3.853376402s" totalImagesPullingTime="3.02920474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:40:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:40:41.851604395 +0000 UTC m=+278.570953935" watchObservedRunningTime="2026-07-02 21:40:41.853376402 +0000 UTC m=+278.572725945" Jul 02 21:40:48.365664 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:48.365621 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:40:48.471962 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:48.471923 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:40:48.978856 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:48.978820 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:40:49.143220 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:49.143187 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:40:49.252694 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:49.252623 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:40:49.342114 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:49.342089 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-27xdp"] Jul 02 21:40:49.344734 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:49.344719 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-27xdp" Jul 02 21:40:49.348448 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:49.348424 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgbhh\" (UniqueName: \"kubernetes.io/projected/6360a44c-1e00-45a6-8ba0-42b74f41bba5-kube-api-access-bgbhh\") pod \"opendatahub-operator-controller-manager-bcc4b86cc-27xdp\" (UID: \"6360a44c-1e00-45a6-8ba0-42b74f41bba5\") " pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-27xdp" Jul 02 21:40:49.348557 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:49.348459 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6360a44c-1e00-45a6-8ba0-42b74f41bba5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bcc4b86cc-27xdp\" (UID: \"6360a44c-1e00-45a6-8ba0-42b74f41bba5\") " pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-27xdp" Jul 02 21:40:49.348557 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:49.348494 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6360a44c-1e00-45a6-8ba0-42b74f41bba5-tmp\") pod \"opendatahub-operator-controller-manager-bcc4b86cc-27xdp\" (UID: \"6360a44c-1e00-45a6-8ba0-42b74f41bba5\") " pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-27xdp" Jul 02 21:40:49.348557 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:49.348539 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6360a44c-1e00-45a6-8ba0-42b74f41bba5-webhook-cert\") pod \"opendatahub-operator-controller-manager-bcc4b86cc-27xdp\" (UID: \"6360a44c-1e00-45a6-8ba0-42b74f41bba5\") " pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-27xdp" Jul 02 21:40:49.348682 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:49.348576 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6360a44c-1e00-45a6-8ba0-42b74f41bba5-manifests\") pod \"opendatahub-operator-controller-manager-bcc4b86cc-27xdp\" (UID: \"6360a44c-1e00-45a6-8ba0-42b74f41bba5\") " pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-27xdp" Jul 02 21:40:49.349142 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:49.349125 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 02 21:40:49.349344 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:49.349331 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-gm24m\"" Jul 02 21:40:49.349596 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:49.349581 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 02 21:40:49.349689 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:49.349606 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 02 21:40:49.350511 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:49.350494 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 02 21:40:49.449433 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:49.449399 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgbhh\" (UniqueName: \"kubernetes.io/projected/6360a44c-1e00-45a6-8ba0-42b74f41bba5-kube-api-access-bgbhh\") pod \"opendatahub-operator-controller-manager-bcc4b86cc-27xdp\" (UID: \"6360a44c-1e00-45a6-8ba0-42b74f41bba5\") " pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-27xdp" Jul 02 21:40:49.449859 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:49.449441 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6360a44c-1e00-45a6-8ba0-42b74f41bba5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bcc4b86cc-27xdp\" (UID: \"6360a44c-1e00-45a6-8ba0-42b74f41bba5\") " pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-27xdp" Jul 02 21:40:49.449859 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:49.449481 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6360a44c-1e00-45a6-8ba0-42b74f41bba5-tmp\") pod \"opendatahub-operator-controller-manager-bcc4b86cc-27xdp\" (UID: \"6360a44c-1e00-45a6-8ba0-42b74f41bba5\") " pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-27xdp" Jul 02 21:40:49.449859 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:49.449665 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6360a44c-1e00-45a6-8ba0-42b74f41bba5-webhook-cert\") pod \"opendatahub-operator-controller-manager-bcc4b86cc-27xdp\" (UID: \"6360a44c-1e00-45a6-8ba0-42b74f41bba5\") " pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-27xdp" Jul 02 21:40:49.449859 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:49.449705 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6360a44c-1e00-45a6-8ba0-42b74f41bba5-manifests\") pod \"opendatahub-operator-controller-manager-bcc4b86cc-27xdp\" (UID: \"6360a44c-1e00-45a6-8ba0-42b74f41bba5\") " pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-27xdp" Jul 02 21:40:49.450082 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:49.450056 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6360a44c-1e00-45a6-8ba0-42b74f41bba5-manifests\") pod \"opendatahub-operator-controller-manager-bcc4b86cc-27xdp\" (UID: \"6360a44c-1e00-45a6-8ba0-42b74f41bba5\") " pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-27xdp" Jul 02 21:40:49.452001 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:49.451979 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6360a44c-1e00-45a6-8ba0-42b74f41bba5-tmp\") pod \"opendatahub-operator-controller-manager-bcc4b86cc-27xdp\" (UID: \"6360a44c-1e00-45a6-8ba0-42b74f41bba5\") " pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-27xdp" Jul 02 21:40:49.452116 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:49.452092 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6360a44c-1e00-45a6-8ba0-42b74f41bba5-webhook-cert\") pod \"opendatahub-operator-controller-manager-bcc4b86cc-27xdp\" (UID: \"6360a44c-1e00-45a6-8ba0-42b74f41bba5\") " pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-27xdp" Jul 02 21:40:49.452166 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:49.452098 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6360a44c-1e00-45a6-8ba0-42b74f41bba5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bcc4b86cc-27xdp\" (UID: \"6360a44c-1e00-45a6-8ba0-42b74f41bba5\") " pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-27xdp" Jul 02 21:40:49.467970 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:49.467948 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-27xdp"] Jul 02 21:40:49.475422 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:49.475399 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgbhh\" (UniqueName: \"kubernetes.io/projected/6360a44c-1e00-45a6-8ba0-42b74f41bba5-kube-api-access-bgbhh\") pod \"opendatahub-operator-controller-manager-bcc4b86cc-27xdp\" (UID: \"6360a44c-1e00-45a6-8ba0-42b74f41bba5\") " pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-27xdp" Jul 02 21:40:49.579883 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:49.579830 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:40:49.654273 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:49.654239 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-27xdp" Jul 02 21:40:49.680693 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:49.680660 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:40:49.755247 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:49.755195 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:40:49.788053 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:49.788034 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-27xdp"] Jul 02 21:40:49.790592 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:40:49.790572 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6360a44c_1e00_45a6_8ba0_42b74f41bba5.slice/crio-6adca8e037f7cce8537eeb65a5b097ec980ffc75e9de69418c53d95054f1e76a WatchSource:0}: Error finding container 6adca8e037f7cce8537eeb65a5b097ec980ffc75e9de69418c53d95054f1e76a: Status 404 returned error can't find the container with id 6adca8e037f7cce8537eeb65a5b097ec980ffc75e9de69418c53d95054f1e76a Jul 02 21:40:49.824423 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:49.824400 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:40:49.865059 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:49.865005 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-27xdp" event={"ID":"6360a44c-1e00-45a6-8ba0-42b74f41bba5","Type":"ContainerStarted","Data":"6adca8e037f7cce8537eeb65a5b097ec980ffc75e9de69418c53d95054f1e76a"} Jul 02 21:40:49.925002 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:49.924974 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:40:50.024822 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:50.024768 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:40:52.839251 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:52.839224 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-8677d7b8dc-xtgj6" Jul 02 21:40:52.878212 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:52.878184 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-27xdp" event={"ID":"6360a44c-1e00-45a6-8ba0-42b74f41bba5","Type":"ContainerStarted","Data":"8f3c39c6cea72dc294e4edfdf80901f050ee4f99bba741dd45f9407af91d3be8"} Jul 02 21:40:54.885495 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:54.885463 2556 generic.go:356] "Generic (PLEG): container finished" podID="6360a44c-1e00-45a6-8ba0-42b74f41bba5" containerID="8f3c39c6cea72dc294e4edfdf80901f050ee4f99bba741dd45f9407af91d3be8" exitCode=0 Jul 02 21:40:54.885958 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:54.885508 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-27xdp" event={"ID":"6360a44c-1e00-45a6-8ba0-42b74f41bba5","Type":"ContainerDied","Data":"8f3c39c6cea72dc294e4edfdf80901f050ee4f99bba741dd45f9407af91d3be8"} Jul 02 21:40:55.890488 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:55.890457 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-27xdp" event={"ID":"6360a44c-1e00-45a6-8ba0-42b74f41bba5","Type":"ContainerStarted","Data":"ca04a47d4a0f61f8b129138630aa97a8b02ea2a05ec42ff24df755971ea18a63"} Jul 02 21:40:55.890838 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:55.890654 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-27xdp" Jul 02 21:40:55.916646 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:40:55.916584 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-27xdp" podStartSLOduration=2.49000913 podStartE2EDuration="6.916570076s" totalImagesPullingTime="3.339960437s" totalInitContainerRuntime="1.086600509s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:40:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:40:55.914960551 +0000 UTC m=+292.634310092" watchObservedRunningTime="2026-07-02 21:40:55.916570076 +0000 UTC m=+292.635919613" Jul 02 21:41:03.680179 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:03.680148 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8jx8_7a32947e-1a4e-4138-8367-6def69748311/ovn-acl-logging/0.log" Jul 02 21:41:03.682242 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:03.682216 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8jx8_7a32947e-1a4e-4138-8367-6def69748311/ovn-acl-logging/0.log" Jul 02 21:41:03.684953 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:03.684934 2556 kubelet.go:1648] "Image garbage collection succeeded" Jul 02 21:41:06.896317 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:06.896285 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-bcc4b86cc-27xdp" Jul 02 21:41:08.417945 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:08.417907 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:41:09.724729 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:09.724689 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:41:10.336003 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:10.335961 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:41:10.453427 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:10.453398 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:41:10.525028 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:10.525002 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:41:11.455939 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:11.455911 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:41:16.875753 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:16.875721 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44zd7"] Jul 02 21:41:16.878011 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:16.877996 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44zd7" Jul 02 21:41:16.880601 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:16.880575 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 02 21:41:16.880724 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:16.880678 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 02 21:41:16.881291 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:16.881272 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 02 21:41:16.881380 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:16.881306 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 02 21:41:16.881380 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:16.881310 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 02 21:41:16.881725 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:16.881707 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-f2mq5\"" Jul 02 21:41:16.881833 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:16.881711 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 02 21:41:16.891710 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:16.891689 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44zd7"] Jul 02 21:41:16.950463 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:16.950439 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5256ef8e-3e79-4873-8cde-41997843d404-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-44zd7\" (UID: \"5256ef8e-3e79-4873-8cde-41997843d404\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44zd7" Jul 02 21:41:16.950566 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:16.950468 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5256ef8e-3e79-4873-8cde-41997843d404-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-44zd7\" (UID: \"5256ef8e-3e79-4873-8cde-41997843d404\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44zd7" Jul 02 21:41:16.950566 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:16.950507 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5256ef8e-3e79-4873-8cde-41997843d404-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-44zd7\" (UID: \"5256ef8e-3e79-4873-8cde-41997843d404\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44zd7" Jul 02 21:41:16.950566 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:16.950532 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5256ef8e-3e79-4873-8cde-41997843d404-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-44zd7\" (UID: \"5256ef8e-3e79-4873-8cde-41997843d404\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44zd7" Jul 02 21:41:16.950566 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:16.950561 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6cv5\" (UniqueName: \"kubernetes.io/projected/5256ef8e-3e79-4873-8cde-41997843d404-kube-api-access-s6cv5\") pod \"istiod-openshift-gateway-94bb8fbfd-44zd7\" (UID: \"5256ef8e-3e79-4873-8cde-41997843d404\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44zd7" Jul 02 21:41:16.950697 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:16.950585 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5256ef8e-3e79-4873-8cde-41997843d404-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-44zd7\" (UID: \"5256ef8e-3e79-4873-8cde-41997843d404\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44zd7" Jul 02 21:41:16.950748 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:16.950731 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5256ef8e-3e79-4873-8cde-41997843d404-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-44zd7\" (UID: \"5256ef8e-3e79-4873-8cde-41997843d404\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44zd7" Jul 02 21:41:17.051367 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:17.051336 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5256ef8e-3e79-4873-8cde-41997843d404-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-44zd7\" (UID: \"5256ef8e-3e79-4873-8cde-41997843d404\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44zd7" Jul 02 21:41:17.051498 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:17.051377 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5256ef8e-3e79-4873-8cde-41997843d404-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-44zd7\" (UID: \"5256ef8e-3e79-4873-8cde-41997843d404\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44zd7" Jul 02 21:41:17.051498 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:17.051401 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6cv5\" (UniqueName: \"kubernetes.io/projected/5256ef8e-3e79-4873-8cde-41997843d404-kube-api-access-s6cv5\") pod \"istiod-openshift-gateway-94bb8fbfd-44zd7\" (UID: \"5256ef8e-3e79-4873-8cde-41997843d404\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44zd7" Jul 02 21:41:17.051498 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:17.051423 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5256ef8e-3e79-4873-8cde-41997843d404-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-44zd7\" (UID: \"5256ef8e-3e79-4873-8cde-41997843d404\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44zd7" Jul 02 21:41:17.051498 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:17.051445 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5256ef8e-3e79-4873-8cde-41997843d404-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-44zd7\" (UID: \"5256ef8e-3e79-4873-8cde-41997843d404\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44zd7" Jul 02 21:41:17.051498 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:17.051478 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5256ef8e-3e79-4873-8cde-41997843d404-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-44zd7\" (UID: \"5256ef8e-3e79-4873-8cde-41997843d404\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44zd7" Jul 02 21:41:17.051719 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:17.051500 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5256ef8e-3e79-4873-8cde-41997843d404-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-44zd7\" (UID: \"5256ef8e-3e79-4873-8cde-41997843d404\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44zd7" Jul 02 21:41:17.052378 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:17.052349 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5256ef8e-3e79-4873-8cde-41997843d404-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-44zd7\" (UID: \"5256ef8e-3e79-4873-8cde-41997843d404\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44zd7" Jul 02 21:41:17.053900 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:17.053853 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5256ef8e-3e79-4873-8cde-41997843d404-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-44zd7\" (UID: \"5256ef8e-3e79-4873-8cde-41997843d404\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44zd7" Jul 02 21:41:17.053995 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:17.053898 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5256ef8e-3e79-4873-8cde-41997843d404-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-44zd7\" (UID: \"5256ef8e-3e79-4873-8cde-41997843d404\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44zd7" Jul 02 21:41:17.054034 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:17.053989 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5256ef8e-3e79-4873-8cde-41997843d404-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-44zd7\" (UID: \"5256ef8e-3e79-4873-8cde-41997843d404\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44zd7" Jul 02 21:41:17.054081 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:17.054064 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5256ef8e-3e79-4873-8cde-41997843d404-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-44zd7\" (UID: \"5256ef8e-3e79-4873-8cde-41997843d404\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44zd7" Jul 02 21:41:17.061941 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:17.061915 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5256ef8e-3e79-4873-8cde-41997843d404-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-44zd7\" (UID: \"5256ef8e-3e79-4873-8cde-41997843d404\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44zd7" Jul 02 21:41:17.062597 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:17.062579 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6cv5\" (UniqueName: \"kubernetes.io/projected/5256ef8e-3e79-4873-8cde-41997843d404-kube-api-access-s6cv5\") pod \"istiod-openshift-gateway-94bb8fbfd-44zd7\" (UID: \"5256ef8e-3e79-4873-8cde-41997843d404\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44zd7" Jul 02 21:41:17.186413 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:17.186359 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44zd7" Jul 02 21:41:17.319130 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:17.319096 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44zd7"] Jul 02 21:41:17.327051 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:41:17.327000 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5256ef8e_3e79_4873_8cde_41997843d404.slice/crio-daed124951102aa2510ebf3eb22268ed3a1e36d8734d36c65c0373b86b07c6d6 WatchSource:0}: Error finding container daed124951102aa2510ebf3eb22268ed3a1e36d8734d36c65c0373b86b07c6d6: Status 404 returned error can't find the container with id daed124951102aa2510ebf3eb22268ed3a1e36d8734d36c65c0373b86b07c6d6 Jul 02 21:41:17.328696 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:17.328681 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 21:41:17.802109 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:17.802076 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:41:17.904341 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:17.904292 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:41:17.964681 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:17.964642 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44zd7" event={"ID":"5256ef8e-3e79-4873-8cde-41997843d404","Type":"ContainerStarted","Data":"daed124951102aa2510ebf3eb22268ed3a1e36d8734d36c65c0373b86b07c6d6"} Jul 02 21:41:18.007125 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:18.007094 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:41:18.279304 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:18.279271 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:41:18.447668 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:18.447633 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:41:19.649172 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:19.649139 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 21:41:19.649452 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:19.649201 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 21:41:19.867540 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:19.867501 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:41:19.950099 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:19.950030 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:41:19.973666 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:19.973631 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44zd7" event={"ID":"5256ef8e-3e79-4873-8cde-41997843d404","Type":"ContainerStarted","Data":"41ac3259784af5f7ea4d3e0bf96f44f54d0b5deb01aa3f3473285b7d373c5e5e"} Jul 02 21:41:19.973845 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:19.973719 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44zd7" Jul 02 21:41:20.006401 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:20.006337 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44zd7" podStartSLOduration=1.6861995109999999 podStartE2EDuration="4.006315783s" totalImagesPullingTime="2.320116272s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:41:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:41:20.004388948 +0000 UTC m=+316.723738490" watchObservedRunningTime="2026-07-02 21:41:20.006315783 +0000 UTC m=+316.725665327" Jul 02 21:41:20.332821 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:20.332787 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:41:20.978728 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:20.978698 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-44zd7" Jul 02 21:41:21.104149 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:21.104122 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:41:21.221119 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:21.221093 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:41:21.333781 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:21.333756 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:41:29.938738 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:41:29.938706 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:42:09.884580 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:42:09.884535 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:42:09.982112 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:42:09.982086 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:42:10.082497 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:42:10.082458 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:42:10.827655 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:42:10.827624 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:42:11.025433 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:42:11.025392 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:42:12.385928 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:42:12.385886 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:42:12.451644 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:42:12.451617 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:42:39.491563 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:42:39.491526 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:42:40.298598 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:42:40.298564 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:42:40.750326 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:42:40.750238 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:42:40.927472 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:42:40.927436 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:42:41.487354 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:42:41.487323 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:42:41.583837 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:42:41.583806 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:42:44.712103 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:42:44.712067 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:42:47.154860 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:42:47.154830 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:42:47.303438 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:42:47.303406 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:42:48.161216 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:42:48.161175 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:42:51.297554 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:42:51.297519 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:42:51.941595 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:42:51.941567 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:42:52.886346 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:42:52.886314 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:42:53.174025 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:42:53.173943 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:42:53.258615 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:42:53.258583 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:42:54.897442 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:42:54.897404 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:42:54.997650 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:42:54.997623 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:42:55.498056 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:42:55.498017 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:42:55.598587 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:42:55.598559 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:42:57.905282 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:42:57.905251 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:42:58.008241 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:42:58.008210 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:42:59.350242 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:42:59.350212 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:42:59.447664 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:42:59.447630 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:43:00.700279 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:00.700249 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:43:00.799830 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:00.799798 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:43:05.100066 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:05.100034 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:43:05.200549 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:05.200517 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:43:13.900151 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:13.900115 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:43:13.999765 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:13.999731 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:43:15.354212 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:15.354174 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:43:15.407821 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:15.407800 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:43:17.895445 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:17.895406 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:43:17.995625 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:17.995595 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:43:18.493289 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:18.493252 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:43:18.550478 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:18.550447 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:43:22.909396 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:22.909365 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:43:23.013361 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:23.013328 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:43:26.696704 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:26.696629 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:43:26.797983 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:26.797942 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:43:33.573230 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:33.573196 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:43:33.676711 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:33.676674 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:43:38.523580 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:38.523551 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:43:38.702407 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:38.702368 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:43:41.814070 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:41.814043 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:43:50.894554 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:50.894512 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:43:51.002003 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:51.001973 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:43:51.060316 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:51.060288 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:43:51.210006 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:51.209935 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:43:51.212580 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:51.212565 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-k8cbc" Jul 02 21:43:51.215000 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:51.214976 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 02 21:43:51.215107 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:51.215075 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 02 21:43:51.215107 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:51.215075 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 02 21:43:51.216179 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:51.216161 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-zzx2n\"" Jul 02 21:43:51.223235 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:51.223214 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:43:51.256304 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:51.256278 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:43:51.348640 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:51.348612 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v58r9\" (UniqueName: \"kubernetes.io/projected/9e8af168-3721-4834-adf0-e3e4b4922633-kube-api-access-v58r9\") pod \"limitador-limitador-54d6d988d5-k8cbc\" (UID: \"9e8af168-3721-4834-adf0-e3e4b4922633\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-k8cbc" Jul 02 21:43:51.348756 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:51.348643 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9e8af168-3721-4834-adf0-e3e4b4922633-config-file\") pod \"limitador-limitador-54d6d988d5-k8cbc\" (UID: \"9e8af168-3721-4834-adf0-e3e4b4922633\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-k8cbc" Jul 02 21:43:51.449536 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:51.449507 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v58r9\" (UniqueName: \"kubernetes.io/projected/9e8af168-3721-4834-adf0-e3e4b4922633-kube-api-access-v58r9\") pod \"limitador-limitador-54d6d988d5-k8cbc\" (UID: \"9e8af168-3721-4834-adf0-e3e4b4922633\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-k8cbc" Jul 02 21:43:51.449667 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:51.449541 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9e8af168-3721-4834-adf0-e3e4b4922633-config-file\") pod \"limitador-limitador-54d6d988d5-k8cbc\" (UID: \"9e8af168-3721-4834-adf0-e3e4b4922633\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-k8cbc" Jul 02 21:43:51.450203 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:51.450183 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9e8af168-3721-4834-adf0-e3e4b4922633-config-file\") pod \"limitador-limitador-54d6d988d5-k8cbc\" (UID: \"9e8af168-3721-4834-adf0-e3e4b4922633\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-k8cbc" Jul 02 21:43:51.458395 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:51.458378 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v58r9\" (UniqueName: \"kubernetes.io/projected/9e8af168-3721-4834-adf0-e3e4b4922633-kube-api-access-v58r9\") pod \"limitador-limitador-54d6d988d5-k8cbc\" (UID: \"9e8af168-3721-4834-adf0-e3e4b4922633\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-k8cbc" Jul 02 21:43:51.522440 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:51.522406 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-k8cbc" Jul 02 21:43:51.648746 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:51.648675 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:43:51.651259 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:43:51.651231 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e8af168_3721_4834_adf0_e3e4b4922633.slice/crio-bdfebd2dc5b4d39ca46853b239b365b1650c01cf553d5f822dd9e17864754013 WatchSource:0}: Error finding container bdfebd2dc5b4d39ca46853b239b365b1650c01cf553d5f822dd9e17864754013: Status 404 returned error can't find the container with id bdfebd2dc5b4d39ca46853b239b365b1650c01cf553d5f822dd9e17864754013 Jul 02 21:43:51.730400 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:51.730373 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:43:51.829910 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:51.829831 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:43:52.499987 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:52.499911 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-k8cbc" event={"ID":"9e8af168-3721-4834-adf0-e3e4b4922633","Type":"ContainerStarted","Data":"bdfebd2dc5b4d39ca46853b239b365b1650c01cf553d5f822dd9e17864754013"} Jul 02 21:43:53.129520 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:53.129491 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:43:53.231265 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:53.231237 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:43:54.508291 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:54.508263 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-k8cbc" event={"ID":"9e8af168-3721-4834-adf0-e3e4b4922633","Type":"ContainerStarted","Data":"69da31692f6be1b7ae1da05486e0b809825620d40372a0cc8602b799c65192a4"} Jul 02 21:43:54.508641 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:54.508383 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-k8cbc" Jul 02 21:43:54.525990 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:43:54.525951 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-k8cbc" podStartSLOduration=1.066789206 podStartE2EDuration="3.525938434s" totalImagesPullingTime="2.459149228s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:43:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:43:54.524737298 +0000 UTC m=+471.244086840" watchObservedRunningTime="2026-07-02 21:43:54.525938434 +0000 UTC m=+471.245287975" Jul 02 21:44:03.674535 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:44:03.674505 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:44:03.774786 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:44:03.774759 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:44:05.513847 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:44:05.513820 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-k8cbc" Jul 02 21:44:12.087373 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:44:12.087338 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:44:51.889577 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:44:51.889541 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:44:52.266268 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:44:52.266232 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:44:58.783292 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:44:58.783207 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:44:58.877085 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:44:58.877050 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:44:58.960677 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:44:58.960644 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:45:01.393715 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:45:01.393682 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:45:01.541928 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:45:01.541848 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:45:25.320010 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:45:25.319976 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:45:33.429759 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:45:33.429725 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:45:33.957171 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:45:33.957134 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:45:34.381363 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:45:34.381332 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:45:34.485116 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:45:34.485086 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:45:34.612478 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:45:34.612442 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:45:44.661396 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:45:44.661359 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:45:44.759495 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:45:44.759469 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:45:49.872336 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:45:49.872305 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:45:50.432486 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:45:50.432451 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:45:56.160538 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:45:56.160502 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:45:56.830726 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:45:56.830693 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:46:03.701582 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:03.701559 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8jx8_7a32947e-1a4e-4138-8367-6def69748311/ovn-acl-logging/0.log" Jul 02 21:46:03.702083 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:03.701561 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8jx8_7a32947e-1a4e-4138-8367-6def69748311/ovn-acl-logging/0.log" Jul 02 21:46:19.883692 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:19.883659 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:46:21.129291 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:21.129263 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:46:24.705447 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:24.705371 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:46:31.634270 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:31.634236 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8"] Jul 02 21:46:31.637166 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:31.637149 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8" Jul 02 21:46:31.639834 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:31.639816 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 02 21:46:31.641060 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:31.641045 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-m6vlx\"" Jul 02 21:46:31.641196 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:31.641176 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 02 21:46:31.641258 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:31.641179 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 02 21:46:31.650548 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:31.650527 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8"] Jul 02 21:46:31.743598 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:31.743563 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:46:31.753644 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:31.753621 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d5bfc38-0ccf-4559-aa72-ff33de61fef8-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8\" (UID: \"1d5bfc38-0ccf-4559-aa72-ff33de61fef8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8" Jul 02 21:46:31.753778 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:31.753667 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d5bfc38-0ccf-4559-aa72-ff33de61fef8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8\" (UID: \"1d5bfc38-0ccf-4559-aa72-ff33de61fef8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8" Jul 02 21:46:31.753778 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:31.753703 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d5bfc38-0ccf-4559-aa72-ff33de61fef8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8\" (UID: \"1d5bfc38-0ccf-4559-aa72-ff33de61fef8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8" Jul 02 21:46:31.753778 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:31.753765 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d5bfc38-0ccf-4559-aa72-ff33de61fef8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8\" (UID: \"1d5bfc38-0ccf-4559-aa72-ff33de61fef8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8" Jul 02 21:46:31.753944 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:31.753795 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfvbk\" (UniqueName: \"kubernetes.io/projected/1d5bfc38-0ccf-4559-aa72-ff33de61fef8-kube-api-access-xfvbk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8\" (UID: \"1d5bfc38-0ccf-4559-aa72-ff33de61fef8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8" Jul 02 21:46:31.753944 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:31.753824 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d5bfc38-0ccf-4559-aa72-ff33de61fef8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8\" (UID: \"1d5bfc38-0ccf-4559-aa72-ff33de61fef8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8" Jul 02 21:46:31.854659 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:31.854634 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d5bfc38-0ccf-4559-aa72-ff33de61fef8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8\" (UID: \"1d5bfc38-0ccf-4559-aa72-ff33de61fef8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8" Jul 02 21:46:31.854823 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:31.854666 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d5bfc38-0ccf-4559-aa72-ff33de61fef8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8\" (UID: \"1d5bfc38-0ccf-4559-aa72-ff33de61fef8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8" Jul 02 21:46:31.854823 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:31.854702 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d5bfc38-0ccf-4559-aa72-ff33de61fef8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8\" (UID: \"1d5bfc38-0ccf-4559-aa72-ff33de61fef8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8" Jul 02 21:46:31.854823 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:31.854721 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfvbk\" (UniqueName: \"kubernetes.io/projected/1d5bfc38-0ccf-4559-aa72-ff33de61fef8-kube-api-access-xfvbk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8\" (UID: \"1d5bfc38-0ccf-4559-aa72-ff33de61fef8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8" Jul 02 21:46:31.854823 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:31.854746 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d5bfc38-0ccf-4559-aa72-ff33de61fef8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8\" (UID: \"1d5bfc38-0ccf-4559-aa72-ff33de61fef8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8" Jul 02 21:46:31.854823 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:31.854799 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d5bfc38-0ccf-4559-aa72-ff33de61fef8-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8\" (UID: \"1d5bfc38-0ccf-4559-aa72-ff33de61fef8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8" Jul 02 21:46:31.855206 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:31.855176 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d5bfc38-0ccf-4559-aa72-ff33de61fef8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8\" (UID: \"1d5bfc38-0ccf-4559-aa72-ff33de61fef8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8" Jul 02 21:46:31.855316 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:31.855228 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d5bfc38-0ccf-4559-aa72-ff33de61fef8-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8\" (UID: \"1d5bfc38-0ccf-4559-aa72-ff33de61fef8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8" Jul 02 21:46:31.855316 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:31.855298 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d5bfc38-0ccf-4559-aa72-ff33de61fef8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8\" (UID: \"1d5bfc38-0ccf-4559-aa72-ff33de61fef8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8" Jul 02 21:46:31.856997 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:31.856974 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d5bfc38-0ccf-4559-aa72-ff33de61fef8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8\" (UID: \"1d5bfc38-0ccf-4559-aa72-ff33de61fef8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8" Jul 02 21:46:31.857287 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:31.857268 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d5bfc38-0ccf-4559-aa72-ff33de61fef8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8\" (UID: \"1d5bfc38-0ccf-4559-aa72-ff33de61fef8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8" Jul 02 21:46:31.863500 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:31.863480 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfvbk\" (UniqueName: \"kubernetes.io/projected/1d5bfc38-0ccf-4559-aa72-ff33de61fef8-kube-api-access-xfvbk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8\" (UID: \"1d5bfc38-0ccf-4559-aa72-ff33de61fef8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8" Jul 02 21:46:31.947536 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:31.947463 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8" Jul 02 21:46:32.082480 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:32.082452 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8"] Jul 02 21:46:32.086244 ip-10-0-140-146 kubenswrapper[2556]: W0702 21:46:32.086217 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d5bfc38_0ccf_4559_aa72_ff33de61fef8.slice/crio-a1abda2b966eb3b8ae5ec92456a7639d03818782cb2594ec327fd35778191276 WatchSource:0}: Error finding container a1abda2b966eb3b8ae5ec92456a7639d03818782cb2594ec327fd35778191276: Status 404 returned error can't find the container with id a1abda2b966eb3b8ae5ec92456a7639d03818782cb2594ec327fd35778191276 Jul 02 21:46:32.087623 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:32.087605 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 21:46:33.051661 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:33.051630 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8" event={"ID":"1d5bfc38-0ccf-4559-aa72-ff33de61fef8","Type":"ContainerStarted","Data":"a1abda2b966eb3b8ae5ec92456a7639d03818782cb2594ec327fd35778191276"} Jul 02 21:46:33.732288 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:33.732258 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:46:34.056398 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:34.056362 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8" event={"ID":"1d5bfc38-0ccf-4559-aa72-ff33de61fef8","Type":"ContainerStarted","Data":"204c8df9f06c54a9be5f84329bfa29a52e619ebb70613bd8fe62369914279bc9"} Jul 02 21:46:34.074341 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:34.074300 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8" podStartSLOduration=2.086034402 podStartE2EDuration="3.074284591s" totalImagesPullingTime="988.250189ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:46:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:46:34.073455436 +0000 UTC m=+630.792805002" watchObservedRunningTime="2026-07-02 21:46:34.074284591 +0000 UTC m=+630.793634132" Jul 02 21:46:41.948286 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:41.948250 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8" Jul 02 21:46:41.948742 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:41.948345 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8" Jul 02 21:46:41.960930 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:41.960907 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8" Jul 02 21:46:42.095935 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:42.095911 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8" Jul 02 21:46:51.366123 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:51.366089 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:46:53.634382 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:46:53.634344 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:47:22.879881 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:47:22.879833 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:47:22.941735 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:47:22.941709 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:47:24.656735 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:47:24.656701 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:47:26.933190 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:47:26.933156 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:48:19.047484 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:48:19.047449 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:48:33.862610 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:48:33.862577 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:48:33.944743 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:48:33.944713 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:48:34.984103 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:48:34.984068 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:48:35.089740 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:48:35.089714 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:49:03.543633 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:49:03.543605 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:49:13.028379 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:49:13.028350 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:49:24.842744 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:49:24.842666 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:49:33.237157 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:49:33.237123 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:49:40.229068 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:49:40.229033 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:49:42.076505 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:49:42.076472 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:49:42.139073 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:49:42.139042 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:49:44.427155 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:49:44.427117 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:49:46.533418 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:49:46.533379 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:49:47.804883 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:49:47.804833 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:50:10.528064 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:50:10.528035 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:50:19.827665 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:50:19.827634 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:50:23.277197 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:50:23.277166 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:50:23.403324 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:50:23.403290 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:50:23.576024 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:50:23.575944 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:50:29.542195 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:50:29.542158 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:50:29.664379 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:50:29.664346 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:50:29.829639 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:50:29.829564 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:50:29.912953 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:50:29.912913 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:50:35.589660 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:50:35.589629 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:50:35.671078 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:50:35.671047 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:50:52.034059 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:50:52.034030 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:51:02.043014 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:51:02.042979 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:51:03.724741 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:51:03.724715 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8jx8_7a32947e-1a4e-4138-8367-6def69748311/ovn-acl-logging/0.log" Jul 02 21:51:03.725406 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:51:03.725384 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8jx8_7a32947e-1a4e-4138-8367-6def69748311/ovn-acl-logging/0.log" Jul 02 21:51:11.830344 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:51:11.830312 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:51:19.733330 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:51:19.733295 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:51:29.940713 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:51:29.940672 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:51:39.336384 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:51:39.336349 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:51:51.033121 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:51:51.033090 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:52:00.631841 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:52:00.631809 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:52:23.545103 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:52:23.545063 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:52:32.532267 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:52:32.532186 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:52:56.342659 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:52:56.342627 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:53:06.728434 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:53:06.728395 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:53:20.643204 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:53:20.643172 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:53:30.536287 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:53:30.536257 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:53:31.539316 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:53:31.539275 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:53:39.939262 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:53:39.939223 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:53:49.338038 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:53:49.338004 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:53:57.340698 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:53:57.340610 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:55:51.032621 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:55:51.032593 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:56:00.228821 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:56:00.228786 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:56:03.744517 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:56:03.744491 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8jx8_7a32947e-1a4e-4138-8367-6def69748311/ovn-acl-logging/0.log" Jul 02 21:56:03.745409 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:56:03.745390 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8jx8_7a32947e-1a4e-4138-8367-6def69748311/ovn-acl-logging/0.log" Jul 02 21:56:13.528015 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:56:13.527984 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:56:13.719432 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:56:13.719401 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:56:22.842030 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:56:22.841999 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:56:39.951422 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:56:39.951396 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:56:44.416895 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:56:44.416838 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:56:50.636209 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:56:50.636172 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:57:01.032648 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:57:01.032572 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:57:10.131917 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:57:10.131880 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:57:20.851376 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:57:20.851344 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:57:32.546782 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:57:32.546752 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:57:41.636207 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:57:41.636169 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:57:58.528928 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:57:58.528891 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:58:07.530028 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:58:07.529985 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:58:19.239886 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:58:19.239847 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:58:29.033816 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:58:29.033723 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:58:37.535819 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:58:37.535783 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:58:50.536161 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:58:50.536132 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:59:04.539357 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:59:04.539323 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:59:13.642583 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:59:13.642548 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:59:27.841474 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:59:27.841443 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:59:35.520320 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:59:35.520292 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 21:59:37.227236 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:59:37.227202 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:59:50.535572 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:59:50.535535 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 21:59:58.432697 ip-10-0-140-146 kubenswrapper[2556]: I0702 21:59:58.432620 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:00:00.145746 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:00:00.145712 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29717160-lg8nj"] Jul 02 22:00:00.148810 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:00:00.148793 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717160-lg8nj" Jul 02 22:00:00.152094 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:00:00.152074 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-66gqh\"" Jul 02 22:00:00.168529 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:00:00.168510 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29717160-lg8nj"] Jul 02 22:00:00.229111 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:00:00.229079 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fb2q\" (UniqueName: \"kubernetes.io/projected/80226df1-8f5a-42ce-8d75-280a5cb5fab8-kube-api-access-5fb2q\") pod \"maas-api-key-cleanup-29717160-lg8nj\" (UID: \"80226df1-8f5a-42ce-8d75-280a5cb5fab8\") " pod="opendatahub/maas-api-key-cleanup-29717160-lg8nj" Jul 02 22:00:00.330410 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:00:00.330384 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fb2q\" (UniqueName: \"kubernetes.io/projected/80226df1-8f5a-42ce-8d75-280a5cb5fab8-kube-api-access-5fb2q\") pod \"maas-api-key-cleanup-29717160-lg8nj\" (UID: \"80226df1-8f5a-42ce-8d75-280a5cb5fab8\") " pod="opendatahub/maas-api-key-cleanup-29717160-lg8nj" Jul 02 22:00:00.339134 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:00:00.339107 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fb2q\" (UniqueName: \"kubernetes.io/projected/80226df1-8f5a-42ce-8d75-280a5cb5fab8-kube-api-access-5fb2q\") pod \"maas-api-key-cleanup-29717160-lg8nj\" (UID: \"80226df1-8f5a-42ce-8d75-280a5cb5fab8\") " pod="opendatahub/maas-api-key-cleanup-29717160-lg8nj" Jul 02 22:00:00.458244 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:00:00.458167 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717160-lg8nj" Jul 02 22:00:00.586989 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:00:00.586956 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29717160-lg8nj"] Jul 02 22:00:00.588649 ip-10-0-140-146 kubenswrapper[2556]: W0702 22:00:00.588609 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80226df1_8f5a_42ce_8d75_280a5cb5fab8.slice/crio-b49fd178127da8eac0cda64b53c0a03dce6fc2a6b88daed94cdce1e232806609 WatchSource:0}: Error finding container b49fd178127da8eac0cda64b53c0a03dce6fc2a6b88daed94cdce1e232806609: Status 404 returned error can't find the container with id b49fd178127da8eac0cda64b53c0a03dce6fc2a6b88daed94cdce1e232806609 Jul 02 22:00:00.590597 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:00:00.590578 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 22:00:00.876817 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:00:00.876781 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717160-lg8nj" event={"ID":"80226df1-8f5a-42ce-8d75-280a5cb5fab8","Type":"ContainerStarted","Data":"b49fd178127da8eac0cda64b53c0a03dce6fc2a6b88daed94cdce1e232806609"} Jul 02 22:00:07.901918 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:00:07.901884 2556 generic.go:356] "Generic (PLEG): container finished" podID="80226df1-8f5a-42ce-8d75-280a5cb5fab8" containerID="31d4dd568d4c8aa51c9eba465ef345de18633e202a5ca9f0a05b0f26f58902db" exitCode=0 Jul 02 22:00:07.902325 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:00:07.901923 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717160-lg8nj" event={"ID":"80226df1-8f5a-42ce-8d75-280a5cb5fab8","Type":"ContainerDied","Data":"31d4dd568d4c8aa51c9eba465ef345de18633e202a5ca9f0a05b0f26f58902db"} Jul 02 22:00:09.024626 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:00:09.024605 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717160-lg8nj" Jul 02 22:00:09.099881 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:00:09.099849 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fb2q\" (UniqueName: \"kubernetes.io/projected/80226df1-8f5a-42ce-8d75-280a5cb5fab8-kube-api-access-5fb2q\") pod \"80226df1-8f5a-42ce-8d75-280a5cb5fab8\" (UID: \"80226df1-8f5a-42ce-8d75-280a5cb5fab8\") " Jul 02 22:00:09.102137 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:00:09.102109 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80226df1-8f5a-42ce-8d75-280a5cb5fab8-kube-api-access-5fb2q" (OuterVolumeSpecName: "kube-api-access-5fb2q") pod "80226df1-8f5a-42ce-8d75-280a5cb5fab8" (UID: "80226df1-8f5a-42ce-8d75-280a5cb5fab8"). InnerVolumeSpecName "kube-api-access-5fb2q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:00:09.200887 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:00:09.200795 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fb2q\" (UniqueName: \"kubernetes.io/projected/80226df1-8f5a-42ce-8d75-280a5cb5fab8-kube-api-access-5fb2q\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:00:09.910224 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:00:09.910192 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717160-lg8nj" event={"ID":"80226df1-8f5a-42ce-8d75-280a5cb5fab8","Type":"ContainerDied","Data":"b49fd178127da8eac0cda64b53c0a03dce6fc2a6b88daed94cdce1e232806609"} Jul 02 22:00:09.910353 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:00:09.910224 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717160-lg8nj" Jul 02 22:00:09.910353 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:00:09.910243 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b49fd178127da8eac0cda64b53c0a03dce6fc2a6b88daed94cdce1e232806609" Jul 02 22:00:09.934563 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:00:09.934536 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:00:19.340791 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:00:19.340750 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:00:26.428305 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:00:26.428269 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:00:31.933785 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:00:31.933742 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:00:39.735700 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:00:39.735667 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:00:52.230305 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:00:52.230271 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:01:03.769441 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:03.769413 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8jx8_7a32947e-1a4e-4138-8367-6def69748311/ovn-acl-logging/0.log" Jul 02 22:01:03.770727 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:03.770709 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8jx8_7a32947e-1a4e-4138-8367-6def69748311/ovn-acl-logging/0.log" Jul 02 22:01:04.644679 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:04.644646 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:01:14.028705 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:14.028674 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:01:16.439569 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:16.439493 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:01:16.584810 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:16.584771 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:01:23.984623 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:23.984562 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:01:24.746210 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:24.746169 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:01:27.543958 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:27.543927 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:01:32.204176 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:32.204142 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:01:36.178301 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:36.178271 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:01:36.320617 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:36.320561 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:01:37.300015 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:37.299986 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:01:40.115716 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:40.115680 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:01:40.303082 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:40.303020 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:01:40.584029 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:40.583973 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:01:46.032278 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:46.032244 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:01:46.376294 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:46.376193 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:01:46.455143 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:46.455108 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:01:46.576816 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:46.576758 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:01:46.674410 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:46.674305 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:01:47.778036 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:47.778006 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:01:47.847007 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:47.846976 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-a9c5b595-58f8f9979-dl7v5"] Jul 02 22:01:47.847229 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:47.847218 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80226df1-8f5a-42ce-8d75-280a5cb5fab8" containerName="cleanup" Jul 02 22:01:47.847271 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:47.847231 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="80226df1-8f5a-42ce-8d75-280a5cb5fab8" containerName="cleanup" Jul 02 22:01:47.847317 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:47.847307 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="80226df1-8f5a-42ce-8d75-280a5cb5fab8" containerName="cleanup" Jul 02 22:01:47.849214 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:47.849199 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-a9c5b595-58f8f9979-dl7v5" Jul 02 22:01:47.851774 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:47.851740 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-a9c5b595-serving-cert\"" Jul 02 22:01:47.851934 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:47.851775 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-66gqh\"" Jul 02 22:01:47.856237 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:47.856205 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:01:47.861694 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:47.861674 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-a9c5b595-58f8f9979-dl7v5"] Jul 02 22:01:47.941653 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:47.941630 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:01:47.998101 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:47.998071 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1afa49f6-a18a-412e-b3d1-cbb7e8295733-openshift-service-ca\") pod \"maas-api-e2e-ait-a9c5b595-58f8f9979-dl7v5\" (UID: \"1afa49f6-a18a-412e-b3d1-cbb7e8295733\") " pod="opendatahub/maas-api-e2e-ait-a9c5b595-58f8f9979-dl7v5" Jul 02 22:01:47.998203 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:47.998117 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1afa49f6-a18a-412e-b3d1-cbb7e8295733-maas-api-tls\") pod \"maas-api-e2e-ait-a9c5b595-58f8f9979-dl7v5\" (UID: \"1afa49f6-a18a-412e-b3d1-cbb7e8295733\") " pod="opendatahub/maas-api-e2e-ait-a9c5b595-58f8f9979-dl7v5" Jul 02 22:01:47.998203 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:47.998142 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8v6s\" (UniqueName: \"kubernetes.io/projected/1afa49f6-a18a-412e-b3d1-cbb7e8295733-kube-api-access-j8v6s\") pod \"maas-api-e2e-ait-a9c5b595-58f8f9979-dl7v5\" (UID: \"1afa49f6-a18a-412e-b3d1-cbb7e8295733\") " pod="opendatahub/maas-api-e2e-ait-a9c5b595-58f8f9979-dl7v5" Jul 02 22:01:48.099157 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:48.099076 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1afa49f6-a18a-412e-b3d1-cbb7e8295733-maas-api-tls\") pod \"maas-api-e2e-ait-a9c5b595-58f8f9979-dl7v5\" (UID: \"1afa49f6-a18a-412e-b3d1-cbb7e8295733\") " pod="opendatahub/maas-api-e2e-ait-a9c5b595-58f8f9979-dl7v5" Jul 02 22:01:48.099157 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:48.099117 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8v6s\" (UniqueName: \"kubernetes.io/projected/1afa49f6-a18a-412e-b3d1-cbb7e8295733-kube-api-access-j8v6s\") pod \"maas-api-e2e-ait-a9c5b595-58f8f9979-dl7v5\" (UID: \"1afa49f6-a18a-412e-b3d1-cbb7e8295733\") " pod="opendatahub/maas-api-e2e-ait-a9c5b595-58f8f9979-dl7v5" Jul 02 22:01:48.099347 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:48.099170 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1afa49f6-a18a-412e-b3d1-cbb7e8295733-openshift-service-ca\") pod \"maas-api-e2e-ait-a9c5b595-58f8f9979-dl7v5\" (UID: \"1afa49f6-a18a-412e-b3d1-cbb7e8295733\") " pod="opendatahub/maas-api-e2e-ait-a9c5b595-58f8f9979-dl7v5" Jul 02 22:01:48.099830 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:48.099805 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1afa49f6-a18a-412e-b3d1-cbb7e8295733-openshift-service-ca\") pod \"maas-api-e2e-ait-a9c5b595-58f8f9979-dl7v5\" (UID: \"1afa49f6-a18a-412e-b3d1-cbb7e8295733\") " pod="opendatahub/maas-api-e2e-ait-a9c5b595-58f8f9979-dl7v5" Jul 02 22:01:48.101559 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:48.101539 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1afa49f6-a18a-412e-b3d1-cbb7e8295733-maas-api-tls\") pod \"maas-api-e2e-ait-a9c5b595-58f8f9979-dl7v5\" (UID: \"1afa49f6-a18a-412e-b3d1-cbb7e8295733\") " pod="opendatahub/maas-api-e2e-ait-a9c5b595-58f8f9979-dl7v5" Jul 02 22:01:48.105609 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:48.105586 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:01:48.107205 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:48.107177 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8v6s\" (UniqueName: \"kubernetes.io/projected/1afa49f6-a18a-412e-b3d1-cbb7e8295733-kube-api-access-j8v6s\") pod \"maas-api-e2e-ait-a9c5b595-58f8f9979-dl7v5\" (UID: \"1afa49f6-a18a-412e-b3d1-cbb7e8295733\") " pod="opendatahub/maas-api-e2e-ait-a9c5b595-58f8f9979-dl7v5" Jul 02 22:01:48.159048 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:48.159026 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-a9c5b595-58f8f9979-dl7v5" Jul 02 22:01:48.288417 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:48.288396 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-a9c5b595-58f8f9979-dl7v5"] Jul 02 22:01:48.290344 ip-10-0-140-146 kubenswrapper[2556]: W0702 22:01:48.290319 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1afa49f6_a18a_412e_b3d1_cbb7e8295733.slice/crio-ef0d340da90921db88079e4ca034ff638f9a388c00ed2f261266140613c3445f WatchSource:0}: Error finding container ef0d340da90921db88079e4ca034ff638f9a388c00ed2f261266140613c3445f: Status 404 returned error can't find the container with id ef0d340da90921db88079e4ca034ff638f9a388c00ed2f261266140613c3445f Jul 02 22:01:48.330785 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:48.330757 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:01:48.433697 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:48.433637 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:01:49.266419 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:49.266377 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-a9c5b595-58f8f9979-dl7v5" event={"ID":"1afa49f6-a18a-412e-b3d1-cbb7e8295733","Type":"ContainerStarted","Data":"ef0d340da90921db88079e4ca034ff638f9a388c00ed2f261266140613c3445f"} Jul 02 22:01:50.156930 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:50.156866 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:01:50.271752 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:50.271719 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-a9c5b595-58f8f9979-dl7v5" event={"ID":"1afa49f6-a18a-412e-b3d1-cbb7e8295733","Type":"ContainerStarted","Data":"bd745358289336e71e0e688165318f0f7d17d9d52595cdbb42324cec1aa8b406"} Jul 02 22:01:50.272143 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:50.271796 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-a9c5b595-58f8f9979-dl7v5" Jul 02 22:01:50.291161 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:50.291117 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-a9c5b595-58f8f9979-dl7v5" podStartSLOduration=1.672376983 podStartE2EDuration="3.291102639s" totalImagesPullingTime="1.618725656s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:01:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:01:50.290465694 +0000 UTC m=+1547.009815229" watchObservedRunningTime="2026-07-02 22:01:50.291102639 +0000 UTC m=+1547.010452181" Jul 02 22:01:52.880258 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:52.880226 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:01:53.062803 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:53.062765 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:01:53.160897 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:53.160789 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:01:53.303497 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:53.303462 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-a9c5b595-58f8f9979-dl7v5"] Jul 02 22:01:53.303716 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:53.303676 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-a9c5b595-58f8f9979-dl7v5" podUID="1afa49f6-a18a-412e-b3d1-cbb7e8295733" containerName="maas-api" containerID="cri-o://bd745358289336e71e0e688165318f0f7d17d9d52595cdbb42324cec1aa8b406" gracePeriod=30 Jul 02 22:01:53.338840 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:53.338818 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:01:53.546265 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:53.546243 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-a9c5b595-58f8f9979-dl7v5" Jul 02 22:01:53.643538 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:53.643508 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1afa49f6-a18a-412e-b3d1-cbb7e8295733-maas-api-tls\") pod \"1afa49f6-a18a-412e-b3d1-cbb7e8295733\" (UID: \"1afa49f6-a18a-412e-b3d1-cbb7e8295733\") " Jul 02 22:01:53.643680 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:53.643559 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j8v6s\" (UniqueName: \"kubernetes.io/projected/1afa49f6-a18a-412e-b3d1-cbb7e8295733-kube-api-access-j8v6s\") pod \"1afa49f6-a18a-412e-b3d1-cbb7e8295733\" (UID: \"1afa49f6-a18a-412e-b3d1-cbb7e8295733\") " Jul 02 22:01:53.643680 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:53.643602 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1afa49f6-a18a-412e-b3d1-cbb7e8295733-openshift-service-ca\") pod \"1afa49f6-a18a-412e-b3d1-cbb7e8295733\" (UID: \"1afa49f6-a18a-412e-b3d1-cbb7e8295733\") " Jul 02 22:01:53.644002 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:53.643977 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1afa49f6-a18a-412e-b3d1-cbb7e8295733-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1afa49f6-a18a-412e-b3d1-cbb7e8295733" (UID: "1afa49f6-a18a-412e-b3d1-cbb7e8295733"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:01:53.645706 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:53.645677 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1afa49f6-a18a-412e-b3d1-cbb7e8295733-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1afa49f6-a18a-412e-b3d1-cbb7e8295733" (UID: "1afa49f6-a18a-412e-b3d1-cbb7e8295733"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 22:01:53.645805 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:53.645736 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1afa49f6-a18a-412e-b3d1-cbb7e8295733-kube-api-access-j8v6s" (OuterVolumeSpecName: "kube-api-access-j8v6s") pod "1afa49f6-a18a-412e-b3d1-cbb7e8295733" (UID: "1afa49f6-a18a-412e-b3d1-cbb7e8295733"). InnerVolumeSpecName "kube-api-access-j8v6s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:01:53.744391 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:53.744307 2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1afa49f6-a18a-412e-b3d1-cbb7e8295733-openshift-service-ca\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:01:53.744391 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:53.744345 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1afa49f6-a18a-412e-b3d1-cbb7e8295733-maas-api-tls\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:01:53.744391 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:53.744358 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j8v6s\" (UniqueName: \"kubernetes.io/projected/1afa49f6-a18a-412e-b3d1-cbb7e8295733-kube-api-access-j8v6s\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:01:54.287928 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:54.287896 2556 generic.go:356] "Generic (PLEG): container finished" podID="1afa49f6-a18a-412e-b3d1-cbb7e8295733" containerID="bd745358289336e71e0e688165318f0f7d17d9d52595cdbb42324cec1aa8b406" exitCode=0 Jul 02 22:01:54.288349 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:54.287936 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-a9c5b595-58f8f9979-dl7v5" Jul 02 22:01:54.288349 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:54.287955 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-a9c5b595-58f8f9979-dl7v5" event={"ID":"1afa49f6-a18a-412e-b3d1-cbb7e8295733","Type":"ContainerDied","Data":"bd745358289336e71e0e688165318f0f7d17d9d52595cdbb42324cec1aa8b406"} Jul 02 22:01:54.288349 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:54.287995 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-a9c5b595-58f8f9979-dl7v5" event={"ID":"1afa49f6-a18a-412e-b3d1-cbb7e8295733","Type":"ContainerDied","Data":"ef0d340da90921db88079e4ca034ff638f9a388c00ed2f261266140613c3445f"} Jul 02 22:01:54.288349 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:54.288018 2556 scope.go:117] "RemoveContainer" containerID="bd745358289336e71e0e688165318f0f7d17d9d52595cdbb42324cec1aa8b406" Jul 02 22:01:54.295974 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:54.295864 2556 scope.go:117] "RemoveContainer" containerID="bd745358289336e71e0e688165318f0f7d17d9d52595cdbb42324cec1aa8b406" Jul 02 22:01:54.296213 ip-10-0-140-146 kubenswrapper[2556]: E0702 22:01:54.296196 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd745358289336e71e0e688165318f0f7d17d9d52595cdbb42324cec1aa8b406\": container with ID starting with bd745358289336e71e0e688165318f0f7d17d9d52595cdbb42324cec1aa8b406 not found: ID does not exist" containerID="bd745358289336e71e0e688165318f0f7d17d9d52595cdbb42324cec1aa8b406" Jul 02 22:01:54.296265 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:54.296225 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd745358289336e71e0e688165318f0f7d17d9d52595cdbb42324cec1aa8b406"} err="failed to get container status \"bd745358289336e71e0e688165318f0f7d17d9d52595cdbb42324cec1aa8b406\": rpc error: code = NotFound desc = could not find container \"bd745358289336e71e0e688165318f0f7d17d9d52595cdbb42324cec1aa8b406\": container with ID starting with bd745358289336e71e0e688165318f0f7d17d9d52595cdbb42324cec1aa8b406 not found: ID does not exist" Jul 02 22:01:54.304675 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:54.304656 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-a9c5b595-58f8f9979-dl7v5"] Jul 02 22:01:54.311363 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:54.311339 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-a9c5b595-58f8f9979-dl7v5"] Jul 02 22:01:55.814739 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:55.814707 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1afa49f6-a18a-412e-b3d1-cbb7e8295733" path="/var/lib/kubelet/pods/1afa49f6-a18a-412e-b3d1-cbb7e8295733/volumes" Jul 02 22:01:56.459216 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:56.459183 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:01:58.921960 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:58.921928 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:01:59.044886 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:59.044834 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:01:59.232228 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:59.232194 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:01:59.328973 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:01:59.328929 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:00.352138 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:00.352085 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:00.424754 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:00.424720 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk"] Jul 02 22:02:00.425161 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:00.425142 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1afa49f6-a18a-412e-b3d1-cbb7e8295733" containerName="maas-api" Jul 02 22:02:00.425209 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:00.425166 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="1afa49f6-a18a-412e-b3d1-cbb7e8295733" containerName="maas-api" Jul 02 22:02:00.425256 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:00.425245 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="1afa49f6-a18a-412e-b3d1-cbb7e8295733" containerName="maas-api" Jul 02 22:02:00.429548 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:00.429527 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk" Jul 02 22:02:00.434051 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:00.434025 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-8bfa68a6-serving-cert\"" Jul 02 22:02:00.434177 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:00.434087 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-66gqh\"" Jul 02 22:02:00.441667 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:00.441643 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk"] Jul 02 22:02:00.450102 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:00.450079 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:00.497911 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:00.497864 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e1c8754-768f-4d36-ac1a-0b55f08646a0-maas-api-tls\") pod \"maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk\" (UID: \"0e1c8754-768f-4d36-ac1a-0b55f08646a0\") " pod="opendatahub/maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk" Jul 02 22:02:00.497999 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:00.497909 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e1c8754-768f-4d36-ac1a-0b55f08646a0-openshift-service-ca\") pod \"maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk\" (UID: \"0e1c8754-768f-4d36-ac1a-0b55f08646a0\") " pod="opendatahub/maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk" Jul 02 22:02:00.497999 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:00.497951 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pfjr\" (UniqueName: \"kubernetes.io/projected/0e1c8754-768f-4d36-ac1a-0b55f08646a0-kube-api-access-8pfjr\") pod \"maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk\" (UID: \"0e1c8754-768f-4d36-ac1a-0b55f08646a0\") " pod="opendatahub/maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk" Jul 02 22:02:00.529847 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:00.529812 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:00.599018 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:00.598994 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e1c8754-768f-4d36-ac1a-0b55f08646a0-maas-api-tls\") pod \"maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk\" (UID: \"0e1c8754-768f-4d36-ac1a-0b55f08646a0\") " pod="opendatahub/maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk" Jul 02 22:02:00.599136 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:00.599024 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e1c8754-768f-4d36-ac1a-0b55f08646a0-openshift-service-ca\") pod \"maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk\" (UID: \"0e1c8754-768f-4d36-ac1a-0b55f08646a0\") " pod="opendatahub/maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk" Jul 02 22:02:00.599136 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:00.599055 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pfjr\" (UniqueName: \"kubernetes.io/projected/0e1c8754-768f-4d36-ac1a-0b55f08646a0-kube-api-access-8pfjr\") pod \"maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk\" (UID: \"0e1c8754-768f-4d36-ac1a-0b55f08646a0\") " pod="opendatahub/maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk" Jul 02 22:02:00.599208 ip-10-0-140-146 kubenswrapper[2556]: E0702 22:02:00.599149 2556 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-8bfa68a6-serving-cert: secret "maas-api-e2e-ait-8bfa68a6-serving-cert" not found Jul 02 22:02:00.599241 ip-10-0-140-146 kubenswrapper[2556]: E0702 22:02:00.599215 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e1c8754-768f-4d36-ac1a-0b55f08646a0-maas-api-tls podName:0e1c8754-768f-4d36-ac1a-0b55f08646a0 nodeName:}" failed. No retries permitted until 2026-07-02 22:02:01.09919682 +0000 UTC m=+1557.818546340 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0e1c8754-768f-4d36-ac1a-0b55f08646a0-maas-api-tls") pod "maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk" (UID: "0e1c8754-768f-4d36-ac1a-0b55f08646a0") : secret "maas-api-e2e-ait-8bfa68a6-serving-cert" not found Jul 02 22:02:00.599585 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:00.599568 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e1c8754-768f-4d36-ac1a-0b55f08646a0-openshift-service-ca\") pod \"maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk\" (UID: \"0e1c8754-768f-4d36-ac1a-0b55f08646a0\") " pod="opendatahub/maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk" Jul 02 22:02:00.610379 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:00.610323 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pfjr\" (UniqueName: \"kubernetes.io/projected/0e1c8754-768f-4d36-ac1a-0b55f08646a0-kube-api-access-8pfjr\") pod \"maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk\" (UID: \"0e1c8754-768f-4d36-ac1a-0b55f08646a0\") " pod="opendatahub/maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk" Jul 02 22:02:00.841668 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:00.841637 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:00.942472 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:00.942375 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:01.042836 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:01.042798 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:01.104775 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:01.104739 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e1c8754-768f-4d36-ac1a-0b55f08646a0-maas-api-tls\") pod \"maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk\" (UID: \"0e1c8754-768f-4d36-ac1a-0b55f08646a0\") " pod="opendatahub/maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk" Jul 02 22:02:01.107414 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:01.107389 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e1c8754-768f-4d36-ac1a-0b55f08646a0-maas-api-tls\") pod \"maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk\" (UID: \"0e1c8754-768f-4d36-ac1a-0b55f08646a0\") " pod="opendatahub/maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk" Jul 02 22:02:01.339297 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:01.339262 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk" Jul 02 22:02:01.466012 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:01.465898 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk"] Jul 02 22:02:01.468661 ip-10-0-140-146 kubenswrapper[2556]: W0702 22:02:01.468629 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e1c8754_768f_4d36_ac1a_0b55f08646a0.slice/crio-cc4bccf74458f2bc8e7f8b8717dea3b6d5a06635f3c6666be8f557bbc6acd04d WatchSource:0}: Error finding container cc4bccf74458f2bc8e7f8b8717dea3b6d5a06635f3c6666be8f557bbc6acd04d: Status 404 returned error can't find the container with id cc4bccf74458f2bc8e7f8b8717dea3b6d5a06635f3c6666be8f557bbc6acd04d Jul 02 22:02:02.322915 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:02.322864 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk" event={"ID":"0e1c8754-768f-4d36-ac1a-0b55f08646a0","Type":"ContainerStarted","Data":"c210db23769dac7d0fe645e9c5d6ace9baad50d211f79d86ecc51d5779ffac57"} Jul 02 22:02:02.323103 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:02.322935 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk" event={"ID":"0e1c8754-768f-4d36-ac1a-0b55f08646a0","Type":"ContainerStarted","Data":"cc4bccf74458f2bc8e7f8b8717dea3b6d5a06635f3c6666be8f557bbc6acd04d"} Jul 02 22:02:02.323103 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:02.323008 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk" Jul 02 22:02:02.343919 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:02.343860 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk" podStartSLOduration=1.865649362 podStartE2EDuration="2.343822575s" totalImagesPullingTime="478.173213ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:02:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:02:02.34202557 +0000 UTC m=+1559.061375109" watchObservedRunningTime="2026-07-02 22:02:02.343822575 +0000 UTC m=+1559.063172122" Jul 02 22:02:02.657735 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:02.657706 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:04.034248 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:04.034219 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:02:05.485630 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:05.485592 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:05.755467 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:05.755397 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk"] Jul 02 22:02:05.755661 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:05.755614 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk" podUID="0e1c8754-768f-4d36-ac1a-0b55f08646a0" containerName="maas-api" containerID="cri-o://c210db23769dac7d0fe645e9c5d6ace9baad50d211f79d86ecc51d5779ffac57" gracePeriod=30 Jul 02 22:02:05.789159 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:05.789136 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:05.988559 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:05.988534 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk" Jul 02 22:02:06.145137 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:06.145113 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8pfjr\" (UniqueName: \"kubernetes.io/projected/0e1c8754-768f-4d36-ac1a-0b55f08646a0-kube-api-access-8pfjr\") pod \"0e1c8754-768f-4d36-ac1a-0b55f08646a0\" (UID: \"0e1c8754-768f-4d36-ac1a-0b55f08646a0\") " Jul 02 22:02:06.145284 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:06.145159 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e1c8754-768f-4d36-ac1a-0b55f08646a0-maas-api-tls\") pod \"0e1c8754-768f-4d36-ac1a-0b55f08646a0\" (UID: \"0e1c8754-768f-4d36-ac1a-0b55f08646a0\") " Jul 02 22:02:06.145284 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:06.145177 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e1c8754-768f-4d36-ac1a-0b55f08646a0-openshift-service-ca\") pod \"0e1c8754-768f-4d36-ac1a-0b55f08646a0\" (UID: \"0e1c8754-768f-4d36-ac1a-0b55f08646a0\") " Jul 02 22:02:06.145497 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:06.145467 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e1c8754-768f-4d36-ac1a-0b55f08646a0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0e1c8754-768f-4d36-ac1a-0b55f08646a0" (UID: "0e1c8754-768f-4d36-ac1a-0b55f08646a0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:02:06.147304 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:06.147271 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e1c8754-768f-4d36-ac1a-0b55f08646a0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0e1c8754-768f-4d36-ac1a-0b55f08646a0" (UID: "0e1c8754-768f-4d36-ac1a-0b55f08646a0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 22:02:06.147380 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:06.147364 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e1c8754-768f-4d36-ac1a-0b55f08646a0-kube-api-access-8pfjr" (OuterVolumeSpecName: "kube-api-access-8pfjr") pod "0e1c8754-768f-4d36-ac1a-0b55f08646a0" (UID: "0e1c8754-768f-4d36-ac1a-0b55f08646a0"). InnerVolumeSpecName "kube-api-access-8pfjr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:02:06.246441 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:06.246415 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e1c8754-768f-4d36-ac1a-0b55f08646a0-maas-api-tls\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:02:06.246441 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:06.246437 2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e1c8754-768f-4d36-ac1a-0b55f08646a0-openshift-service-ca\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:02:06.246583 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:06.246450 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8pfjr\" (UniqueName: \"kubernetes.io/projected/0e1c8754-768f-4d36-ac1a-0b55f08646a0-kube-api-access-8pfjr\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:02:06.339010 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:06.338983 2556 generic.go:356] "Generic (PLEG): container finished" podID="0e1c8754-768f-4d36-ac1a-0b55f08646a0" containerID="c210db23769dac7d0fe645e9c5d6ace9baad50d211f79d86ecc51d5779ffac57" exitCode=0 Jul 02 22:02:06.339134 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:06.339026 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk" Jul 02 22:02:06.339134 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:06.339072 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk" event={"ID":"0e1c8754-768f-4d36-ac1a-0b55f08646a0","Type":"ContainerDied","Data":"c210db23769dac7d0fe645e9c5d6ace9baad50d211f79d86ecc51d5779ffac57"} Jul 02 22:02:06.339134 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:06.339121 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk" event={"ID":"0e1c8754-768f-4d36-ac1a-0b55f08646a0","Type":"ContainerDied","Data":"cc4bccf74458f2bc8e7f8b8717dea3b6d5a06635f3c6666be8f557bbc6acd04d"} Jul 02 22:02:06.339282 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:06.339143 2556 scope.go:117] "RemoveContainer" containerID="c210db23769dac7d0fe645e9c5d6ace9baad50d211f79d86ecc51d5779ffac57" Jul 02 22:02:06.347608 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:06.347593 2556 scope.go:117] "RemoveContainer" containerID="c210db23769dac7d0fe645e9c5d6ace9baad50d211f79d86ecc51d5779ffac57" Jul 02 22:02:06.347832 ip-10-0-140-146 kubenswrapper[2556]: E0702 22:02:06.347816 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c210db23769dac7d0fe645e9c5d6ace9baad50d211f79d86ecc51d5779ffac57\": container with ID starting with c210db23769dac7d0fe645e9c5d6ace9baad50d211f79d86ecc51d5779ffac57 not found: ID does not exist" containerID="c210db23769dac7d0fe645e9c5d6ace9baad50d211f79d86ecc51d5779ffac57" Jul 02 22:02:06.347889 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:06.347839 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c210db23769dac7d0fe645e9c5d6ace9baad50d211f79d86ecc51d5779ffac57"} err="failed to get container status \"c210db23769dac7d0fe645e9c5d6ace9baad50d211f79d86ecc51d5779ffac57\": rpc error: code = NotFound desc = could not find container \"c210db23769dac7d0fe645e9c5d6ace9baad50d211f79d86ecc51d5779ffac57\": container with ID starting with c210db23769dac7d0fe645e9c5d6ace9baad50d211f79d86ecc51d5779ffac57 not found: ID does not exist" Jul 02 22:02:06.359883 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:06.359846 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk"] Jul 02 22:02:06.364049 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:06.364029 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-8bfa68a6-68649895f8-2t4bk"] Jul 02 22:02:07.814679 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:07.814643 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e1c8754-768f-4d36-ac1a-0b55f08646a0" path="/var/lib/kubelet/pods/0e1c8754-768f-4d36-ac1a-0b55f08646a0/volumes" Jul 02 22:02:11.262357 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:11.262329 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:11.870526 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:11.870493 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:11.973995 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:11.973962 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:15.826128 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:15.826094 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:02:17.717374 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:17.717341 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:17.828541 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:17.828509 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:17.939696 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:17.939653 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:19.134277 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:19.134242 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:19.180809 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:19.180775 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz"] Jul 02 22:02:19.181100 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:19.181086 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e1c8754-768f-4d36-ac1a-0b55f08646a0" containerName="maas-api" Jul 02 22:02:19.181143 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:19.181102 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e1c8754-768f-4d36-ac1a-0b55f08646a0" containerName="maas-api" Jul 02 22:02:19.181177 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:19.181158 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e1c8754-768f-4d36-ac1a-0b55f08646a0" containerName="maas-api" Jul 02 22:02:19.185227 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:19.185210 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz" Jul 02 22:02:19.187654 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:19.187628 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-e6c9d74c-serving-cert\"" Jul 02 22:02:19.187768 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:19.187751 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-66gqh\"" Jul 02 22:02:19.196510 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:19.196487 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz"] Jul 02 22:02:19.221604 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:19.221579 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:19.245295 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:19.245267 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65qw6\" (UniqueName: \"kubernetes.io/projected/8e7b118d-efa5-4df5-a3e3-f3dd002bfca2-kube-api-access-65qw6\") pod \"maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz\" (UID: \"8e7b118d-efa5-4df5-a3e3-f3dd002bfca2\") " pod="opendatahub/maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz" Jul 02 22:02:19.245441 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:19.245421 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e7b118d-efa5-4df5-a3e3-f3dd002bfca2-maas-api-tls\") pod \"maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz\" (UID: \"8e7b118d-efa5-4df5-a3e3-f3dd002bfca2\") " pod="opendatahub/maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz" Jul 02 22:02:19.245504 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:19.245467 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e7b118d-efa5-4df5-a3e3-f3dd002bfca2-openshift-service-ca\") pod \"maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz\" (UID: \"8e7b118d-efa5-4df5-a3e3-f3dd002bfca2\") " pod="opendatahub/maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz" Jul 02 22:02:19.301117 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:19.301089 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:19.346182 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:19.346152 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e7b118d-efa5-4df5-a3e3-f3dd002bfca2-maas-api-tls\") pod \"maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz\" (UID: \"8e7b118d-efa5-4df5-a3e3-f3dd002bfca2\") " pod="opendatahub/maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz" Jul 02 22:02:19.346332 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:19.346199 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e7b118d-efa5-4df5-a3e3-f3dd002bfca2-openshift-service-ca\") pod \"maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz\" (UID: \"8e7b118d-efa5-4df5-a3e3-f3dd002bfca2\") " pod="opendatahub/maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz" Jul 02 22:02:19.346332 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:19.346241 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65qw6\" (UniqueName: \"kubernetes.io/projected/8e7b118d-efa5-4df5-a3e3-f3dd002bfca2-kube-api-access-65qw6\") pod \"maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz\" (UID: \"8e7b118d-efa5-4df5-a3e3-f3dd002bfca2\") " pod="opendatahub/maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz" Jul 02 22:02:19.346332 ip-10-0-140-146 kubenswrapper[2556]: E0702 22:02:19.346295 2556 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-derive-e6c9d74c-serving-cert: secret "maas-api-e2e-derive-e6c9d74c-serving-cert" not found Jul 02 22:02:19.346481 ip-10-0-140-146 kubenswrapper[2556]: E0702 22:02:19.346366 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e7b118d-efa5-4df5-a3e3-f3dd002bfca2-maas-api-tls podName:8e7b118d-efa5-4df5-a3e3-f3dd002bfca2 nodeName:}" failed. No retries permitted until 2026-07-02 22:02:19.846347838 +0000 UTC m=+1576.565697357 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8e7b118d-efa5-4df5-a3e3-f3dd002bfca2-maas-api-tls") pod "maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz" (UID: "8e7b118d-efa5-4df5-a3e3-f3dd002bfca2") : secret "maas-api-e2e-derive-e6c9d74c-serving-cert" not found Jul 02 22:02:19.346976 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:19.346950 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e7b118d-efa5-4df5-a3e3-f3dd002bfca2-openshift-service-ca\") pod \"maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz\" (UID: \"8e7b118d-efa5-4df5-a3e3-f3dd002bfca2\") " pod="opendatahub/maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz" Jul 02 22:02:19.356863 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:19.356841 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-65qw6\" (UniqueName: \"kubernetes.io/projected/8e7b118d-efa5-4df5-a3e3-f3dd002bfca2-kube-api-access-65qw6\") pod \"maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz\" (UID: \"8e7b118d-efa5-4df5-a3e3-f3dd002bfca2\") " pod="opendatahub/maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz" Jul 02 22:02:19.404742 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:19.404684 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:19.522228 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:19.522205 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:19.686108 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:19.686022 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:19.795439 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:19.795400 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:19.850375 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:19.850347 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e7b118d-efa5-4df5-a3e3-f3dd002bfca2-maas-api-tls\") pod \"maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz\" (UID: \"8e7b118d-efa5-4df5-a3e3-f3dd002bfca2\") " pod="opendatahub/maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz" Jul 02 22:02:19.852725 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:19.852702 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e7b118d-efa5-4df5-a3e3-f3dd002bfca2-maas-api-tls\") pod \"maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz\" (UID: \"8e7b118d-efa5-4df5-a3e3-f3dd002bfca2\") " pod="opendatahub/maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz" Jul 02 22:02:20.095675 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:20.095648 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz" Jul 02 22:02:20.231993 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:20.231970 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz"] Jul 02 22:02:20.234373 ip-10-0-140-146 kubenswrapper[2556]: W0702 22:02:20.234347 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e7b118d_efa5_4df5_a3e3_f3dd002bfca2.slice/crio-02374869b45f7b342d12e00e749929c1b202433da98d184be770bc04c395b960 WatchSource:0}: Error finding container 02374869b45f7b342d12e00e749929c1b202433da98d184be770bc04c395b960: Status 404 returned error can't find the container with id 02374869b45f7b342d12e00e749929c1b202433da98d184be770bc04c395b960 Jul 02 22:02:20.395150 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:20.395073 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz" event={"ID":"8e7b118d-efa5-4df5-a3e3-f3dd002bfca2","Type":"ContainerStarted","Data":"02374869b45f7b342d12e00e749929c1b202433da98d184be770bc04c395b960"} Jul 02 22:02:21.400352 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:21.400316 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz" event={"ID":"8e7b118d-efa5-4df5-a3e3-f3dd002bfca2","Type":"ContainerStarted","Data":"dbc99cdb61678fc0c581375e4cd9f5861bc091a48b733140a946b6faab6b350a"} Jul 02 22:02:21.400741 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:21.400455 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz" Jul 02 22:02:21.421275 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:21.421234 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz" podStartSLOduration=2.086367745 podStartE2EDuration="2.421219548s" totalImagesPullingTime="334.851803ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:02:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:02:21.419926068 +0000 UTC m=+1578.139275606" watchObservedRunningTime="2026-07-02 22:02:21.421219548 +0000 UTC m=+1578.140569096" Jul 02 22:02:22.418269 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:22.418243 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:23.576460 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:23.576424 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:23.745564 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:23.745535 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:23.849119 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:23.849047 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:23.876162 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:23.876136 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz"] Jul 02 22:02:23.876320 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:23.876298 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz" podUID="8e7b118d-efa5-4df5-a3e3-f3dd002bfca2" containerName="maas-api" containerID="cri-o://dbc99cdb61678fc0c581375e4cd9f5861bc091a48b733140a946b6faab6b350a" gracePeriod=30 Jul 02 22:02:23.949919 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:23.949897 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:24.117786 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:24.117757 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz" Jul 02 22:02:24.181378 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:24.181349 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e7b118d-efa5-4df5-a3e3-f3dd002bfca2-openshift-service-ca\") pod \"8e7b118d-efa5-4df5-a3e3-f3dd002bfca2\" (UID: \"8e7b118d-efa5-4df5-a3e3-f3dd002bfca2\") " Jul 02 22:02:24.181517 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:24.181408 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-65qw6\" (UniqueName: \"kubernetes.io/projected/8e7b118d-efa5-4df5-a3e3-f3dd002bfca2-kube-api-access-65qw6\") pod \"8e7b118d-efa5-4df5-a3e3-f3dd002bfca2\" (UID: \"8e7b118d-efa5-4df5-a3e3-f3dd002bfca2\") " Jul 02 22:02:24.181517 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:24.181463 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e7b118d-efa5-4df5-a3e3-f3dd002bfca2-maas-api-tls\") pod \"8e7b118d-efa5-4df5-a3e3-f3dd002bfca2\" (UID: \"8e7b118d-efa5-4df5-a3e3-f3dd002bfca2\") " Jul 02 22:02:24.181680 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:24.181657 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e7b118d-efa5-4df5-a3e3-f3dd002bfca2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8e7b118d-efa5-4df5-a3e3-f3dd002bfca2" (UID: "8e7b118d-efa5-4df5-a3e3-f3dd002bfca2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:02:24.183505 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:24.183480 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e7b118d-efa5-4df5-a3e3-f3dd002bfca2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8e7b118d-efa5-4df5-a3e3-f3dd002bfca2" (UID: "8e7b118d-efa5-4df5-a3e3-f3dd002bfca2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 22:02:24.183596 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:24.183571 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e7b118d-efa5-4df5-a3e3-f3dd002bfca2-kube-api-access-65qw6" (OuterVolumeSpecName: "kube-api-access-65qw6") pod "8e7b118d-efa5-4df5-a3e3-f3dd002bfca2" (UID: "8e7b118d-efa5-4df5-a3e3-f3dd002bfca2"). InnerVolumeSpecName "kube-api-access-65qw6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:02:24.282823 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:24.282801 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e7b118d-efa5-4df5-a3e3-f3dd002bfca2-maas-api-tls\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:02:24.282823 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:24.282821 2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e7b118d-efa5-4df5-a3e3-f3dd002bfca2-openshift-service-ca\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:02:24.282967 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:24.282831 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-65qw6\" (UniqueName: \"kubernetes.io/projected/8e7b118d-efa5-4df5-a3e3-f3dd002bfca2-kube-api-access-65qw6\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:02:24.416519 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:24.416445 2556 generic.go:356] "Generic (PLEG): container finished" podID="8e7b118d-efa5-4df5-a3e3-f3dd002bfca2" containerID="dbc99cdb61678fc0c581375e4cd9f5861bc091a48b733140a946b6faab6b350a" exitCode=0 Jul 02 22:02:24.416519 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:24.416483 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz" Jul 02 22:02:24.416519 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:24.416489 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz" event={"ID":"8e7b118d-efa5-4df5-a3e3-f3dd002bfca2","Type":"ContainerDied","Data":"dbc99cdb61678fc0c581375e4cd9f5861bc091a48b733140a946b6faab6b350a"} Jul 02 22:02:24.416697 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:24.416569 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz" event={"ID":"8e7b118d-efa5-4df5-a3e3-f3dd002bfca2","Type":"ContainerDied","Data":"02374869b45f7b342d12e00e749929c1b202433da98d184be770bc04c395b960"} Jul 02 22:02:24.416697 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:24.416594 2556 scope.go:117] "RemoveContainer" containerID="dbc99cdb61678fc0c581375e4cd9f5861bc091a48b733140a946b6faab6b350a" Jul 02 22:02:24.425341 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:24.425324 2556 scope.go:117] "RemoveContainer" containerID="dbc99cdb61678fc0c581375e4cd9f5861bc091a48b733140a946b6faab6b350a" Jul 02 22:02:24.425558 ip-10-0-140-146 kubenswrapper[2556]: E0702 22:02:24.425543 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbc99cdb61678fc0c581375e4cd9f5861bc091a48b733140a946b6faab6b350a\": container with ID starting with dbc99cdb61678fc0c581375e4cd9f5861bc091a48b733140a946b6faab6b350a not found: ID does not exist" containerID="dbc99cdb61678fc0c581375e4cd9f5861bc091a48b733140a946b6faab6b350a" Jul 02 22:02:24.425607 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:24.425565 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbc99cdb61678fc0c581375e4cd9f5861bc091a48b733140a946b6faab6b350a"} err="failed to get container status \"dbc99cdb61678fc0c581375e4cd9f5861bc091a48b733140a946b6faab6b350a\": rpc error: code = NotFound desc = could not find container \"dbc99cdb61678fc0c581375e4cd9f5861bc091a48b733140a946b6faab6b350a\": container with ID starting with dbc99cdb61678fc0c581375e4cd9f5861bc091a48b733140a946b6faab6b350a not found: ID does not exist" Jul 02 22:02:24.440289 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:24.440263 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz"] Jul 02 22:02:24.446067 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:24.446050 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-e6c9d74c-bf85dd5f4-f5skz"] Jul 02 22:02:25.401836 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:25.401810 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:25.815025 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:25.814997 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e7b118d-efa5-4df5-a3e3-f3dd002bfca2" path="/var/lib/kubelet/pods/8e7b118d-efa5-4df5-a3e3-f3dd002bfca2/volumes" Jul 02 22:02:29.736400 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:29.736372 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:29.837187 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:29.837155 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:30.195728 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:30.195682 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:30.293522 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:30.293491 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:30.415146 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:30.415112 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:31.815614 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:31.815586 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:37.668031 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:37.667995 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:37.773407 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:37.773373 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:37.868140 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:37.868110 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:37.999051 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:37.998970 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:02:38.382409 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:02:38.382377 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:03:04.391254 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:03:04.391220 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:03:10.152027 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:03:10.151993 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:03:10.265645 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:03:10.265602 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:03:10.393233 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:03:10.393201 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:03:26.329932 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:03:26.329895 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:03:32.350800 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:03:32.350765 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:03:32.460986 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:03:32.460954 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:03:32.567735 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:03:32.567703 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:03:43.099815 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:03:43.099781 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:03:43.925072 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:03:43.925036 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:03:50.109737 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:03:50.109699 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:03:50.183014 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:03:50.182986 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:03:50.266005 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:03:50.265968 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:03:50.366850 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:03:50.366781 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:03:50.464045 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:03:50.464018 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:03:51.211310 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:03:51.211267 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:03:51.305528 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:03:51.305499 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:03:51.403831 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:03:51.403804 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:03:53.095735 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:03:53.095702 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:03:53.337735 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:03:53.337702 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:03:59.025101 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:03:59.025063 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:03:59.128470 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:03:59.128435 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:03:59.688212 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:03:59.688181 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:02.434137 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:02.434100 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:04:05.622392 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:05.622361 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:05.726072 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:05.726038 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:05.869785 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:05.869739 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:06.244840 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:06.244791 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:06.513765 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:06.513736 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:09.745244 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:09.745211 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:09.843106 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:09.843057 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:09.915940 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:09.915911 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc"] Jul 02 22:04:09.916183 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:09.916172 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e7b118d-efa5-4df5-a3e3-f3dd002bfca2" containerName="maas-api" Jul 02 22:04:09.916224 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:09.916185 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e7b118d-efa5-4df5-a3e3-f3dd002bfca2" containerName="maas-api" Jul 02 22:04:09.916258 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:09.916230 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e7b118d-efa5-4df5-a3e3-f3dd002bfca2" containerName="maas-api" Jul 02 22:04:09.918825 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:09.918810 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc" Jul 02 22:04:09.921495 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:09.921477 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-1861b388-serving-cert\"" Jul 02 22:04:09.921598 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:09.921480 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-66gqh\"" Jul 02 22:04:09.934508 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:09.934473 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc"] Jul 02 22:04:09.954624 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:09.954600 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/119616bb-9196-419e-92b7-53ece76e98c4-maas-api-tls\") pod \"maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc\" (UID: \"119616bb-9196-419e-92b7-53ece76e98c4\") " pod="opendatahub/maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc" Jul 02 22:04:09.954724 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:09.954656 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/119616bb-9196-419e-92b7-53ece76e98c4-openshift-service-ca\") pod \"maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc\" (UID: \"119616bb-9196-419e-92b7-53ece76e98c4\") " pod="opendatahub/maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc" Jul 02 22:04:09.954770 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:09.954754 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-272h6\" (UniqueName: \"kubernetes.io/projected/119616bb-9196-419e-92b7-53ece76e98c4-kube-api-access-272h6\") pod \"maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc\" (UID: \"119616bb-9196-419e-92b7-53ece76e98c4\") " pod="opendatahub/maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc" Jul 02 22:04:09.959258 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:09.959239 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:10.055609 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:10.055577 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/119616bb-9196-419e-92b7-53ece76e98c4-maas-api-tls\") pod \"maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc\" (UID: \"119616bb-9196-419e-92b7-53ece76e98c4\") " pod="opendatahub/maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc" Jul 02 22:04:10.055783 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:10.055634 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/119616bb-9196-419e-92b7-53ece76e98c4-openshift-service-ca\") pod \"maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc\" (UID: \"119616bb-9196-419e-92b7-53ece76e98c4\") " pod="opendatahub/maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc" Jul 02 22:04:10.055783 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:10.055694 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-272h6\" (UniqueName: \"kubernetes.io/projected/119616bb-9196-419e-92b7-53ece76e98c4-kube-api-access-272h6\") pod \"maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc\" (UID: \"119616bb-9196-419e-92b7-53ece76e98c4\") " pod="opendatahub/maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc" Jul 02 22:04:10.055783 ip-10-0-140-146 kubenswrapper[2556]: E0702 22:04:10.055735 2556 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-1861b388-serving-cert: secret "maas-api-e2e-mt-1861b388-serving-cert" not found Jul 02 22:04:10.055975 ip-10-0-140-146 kubenswrapper[2556]: E0702 22:04:10.055814 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/119616bb-9196-419e-92b7-53ece76e98c4-maas-api-tls podName:119616bb-9196-419e-92b7-53ece76e98c4 nodeName:}" failed. No retries permitted until 2026-07-02 22:04:10.555790053 +0000 UTC m=+1687.275139572 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/119616bb-9196-419e-92b7-53ece76e98c4-maas-api-tls") pod "maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc" (UID: "119616bb-9196-419e-92b7-53ece76e98c4") : secret "maas-api-e2e-mt-1861b388-serving-cert" not found Jul 02 22:04:10.056297 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:10.056280 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/119616bb-9196-419e-92b7-53ece76e98c4-openshift-service-ca\") pod \"maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc\" (UID: \"119616bb-9196-419e-92b7-53ece76e98c4\") " pod="opendatahub/maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc" Jul 02 22:04:10.065078 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:10.065061 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-272h6\" (UniqueName: \"kubernetes.io/projected/119616bb-9196-419e-92b7-53ece76e98c4-kube-api-access-272h6\") pod \"maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc\" (UID: \"119616bb-9196-419e-92b7-53ece76e98c4\") " pod="opendatahub/maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc" Jul 02 22:04:10.533915 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:10.533864 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:10.560288 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:10.560264 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/119616bb-9196-419e-92b7-53ece76e98c4-maas-api-tls\") pod \"maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc\" (UID: \"119616bb-9196-419e-92b7-53ece76e98c4\") " pod="opendatahub/maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc" Jul 02 22:04:10.560428 ip-10-0-140-146 kubenswrapper[2556]: E0702 22:04:10.560380 2556 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-1861b388-serving-cert: secret "maas-api-e2e-mt-1861b388-serving-cert" not found Jul 02 22:04:10.560428 ip-10-0-140-146 kubenswrapper[2556]: E0702 22:04:10.560426 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/119616bb-9196-419e-92b7-53ece76e98c4-maas-api-tls podName:119616bb-9196-419e-92b7-53ece76e98c4 nodeName:}" failed. No retries permitted until 2026-07-02 22:04:11.560413709 +0000 UTC m=+1688.279763228 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/119616bb-9196-419e-92b7-53ece76e98c4-maas-api-tls") pod "maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc" (UID: "119616bb-9196-419e-92b7-53ece76e98c4") : secret "maas-api-e2e-mt-1861b388-serving-cert" not found Jul 02 22:04:10.632735 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:10.632692 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:10.730727 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:10.730699 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:11.568377 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:11.568329 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/119616bb-9196-419e-92b7-53ece76e98c4-maas-api-tls\") pod \"maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc\" (UID: \"119616bb-9196-419e-92b7-53ece76e98c4\") " pod="opendatahub/maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc" Jul 02 22:04:11.570841 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:11.570823 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/119616bb-9196-419e-92b7-53ece76e98c4-maas-api-tls\") pod \"maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc\" (UID: \"119616bb-9196-419e-92b7-53ece76e98c4\") " pod="opendatahub/maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc" Jul 02 22:04:11.728967 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:11.728933 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc" Jul 02 22:04:11.866715 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:11.866694 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc"] Jul 02 22:04:11.868562 ip-10-0-140-146 kubenswrapper[2556]: W0702 22:04:11.868529 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod119616bb_9196_419e_92b7_53ece76e98c4.slice/crio-be22265cd47112dd339f89d9762165d8f62d0cda944ee075a2dccaccd8b290aa WatchSource:0}: Error finding container be22265cd47112dd339f89d9762165d8f62d0cda944ee075a2dccaccd8b290aa: Status 404 returned error can't find the container with id be22265cd47112dd339f89d9762165d8f62d0cda944ee075a2dccaccd8b290aa Jul 02 22:04:12.816536 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:12.816499 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc" event={"ID":"119616bb-9196-419e-92b7-53ece76e98c4","Type":"ContainerStarted","Data":"f509e104323dce56a1edb7d7c00880bd871ee243834fcc0b27c266106fb8d603"} Jul 02 22:04:12.817002 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:12.816554 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc" Jul 02 22:04:12.817002 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:12.816569 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc" event={"ID":"119616bb-9196-419e-92b7-53ece76e98c4","Type":"ContainerStarted","Data":"be22265cd47112dd339f89d9762165d8f62d0cda944ee075a2dccaccd8b290aa"} Jul 02 22:04:12.836492 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:12.836449 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc" podStartSLOduration=3.379213043 podStartE2EDuration="3.836428535s" totalImagesPullingTime="457.215492ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:04:12.835632699 +0000 UTC m=+1689.554982251" watchObservedRunningTime="2026-07-02 22:04:12.836428535 +0000 UTC m=+1689.555778075" Jul 02 22:04:14.250315 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:14.250276 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:14.640522 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:14.640494 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:04:14.897595 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:14.897483 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:14.997721 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:14.997686 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:15.387900 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:15.387853 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:18.830040 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:18.830015 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc" Jul 02 22:04:21.110800 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:21.110765 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:21.209374 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:21.209344 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:21.556411 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:21.556378 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:21.871915 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:21.871832 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc"] Jul 02 22:04:21.872051 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:21.872023 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc" podUID="119616bb-9196-419e-92b7-53ece76e98c4" containerName="maas-api" containerID="cri-o://f509e104323dce56a1edb7d7c00880bd871ee243834fcc0b27c266106fb8d603" gracePeriod=30 Jul 02 22:04:21.896151 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:21.896129 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:21.997719 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:21.997681 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:22.118768 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:22.118746 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc" Jul 02 22:04:22.150448 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:22.150383 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/119616bb-9196-419e-92b7-53ece76e98c4-maas-api-tls\") pod \"119616bb-9196-419e-92b7-53ece76e98c4\" (UID: \"119616bb-9196-419e-92b7-53ece76e98c4\") " Jul 02 22:04:22.150448 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:22.150443 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-272h6\" (UniqueName: \"kubernetes.io/projected/119616bb-9196-419e-92b7-53ece76e98c4-kube-api-access-272h6\") pod \"119616bb-9196-419e-92b7-53ece76e98c4\" (UID: \"119616bb-9196-419e-92b7-53ece76e98c4\") " Jul 02 22:04:22.150623 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:22.150498 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/119616bb-9196-419e-92b7-53ece76e98c4-openshift-service-ca\") pod \"119616bb-9196-419e-92b7-53ece76e98c4\" (UID: \"119616bb-9196-419e-92b7-53ece76e98c4\") " Jul 02 22:04:22.150880 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:22.150849 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/119616bb-9196-419e-92b7-53ece76e98c4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "119616bb-9196-419e-92b7-53ece76e98c4" (UID: "119616bb-9196-419e-92b7-53ece76e98c4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:04:22.152504 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:22.152483 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/119616bb-9196-419e-92b7-53ece76e98c4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "119616bb-9196-419e-92b7-53ece76e98c4" (UID: "119616bb-9196-419e-92b7-53ece76e98c4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 22:04:22.152701 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:22.152677 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/119616bb-9196-419e-92b7-53ece76e98c4-kube-api-access-272h6" (OuterVolumeSpecName: "kube-api-access-272h6") pod "119616bb-9196-419e-92b7-53ece76e98c4" (UID: "119616bb-9196-419e-92b7-53ece76e98c4"). InnerVolumeSpecName "kube-api-access-272h6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:04:22.251829 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:22.251800 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/119616bb-9196-419e-92b7-53ece76e98c4-maas-api-tls\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:04:22.251829 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:22.251823 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-272h6\" (UniqueName: \"kubernetes.io/projected/119616bb-9196-419e-92b7-53ece76e98c4-kube-api-access-272h6\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:04:22.251829 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:22.251833 2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/119616bb-9196-419e-92b7-53ece76e98c4-openshift-service-ca\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:04:22.860481 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:22.860451 2556 generic.go:356] "Generic (PLEG): container finished" podID="119616bb-9196-419e-92b7-53ece76e98c4" containerID="f509e104323dce56a1edb7d7c00880bd871ee243834fcc0b27c266106fb8d603" exitCode=0 Jul 02 22:04:22.860637 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:22.860485 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc" event={"ID":"119616bb-9196-419e-92b7-53ece76e98c4","Type":"ContainerDied","Data":"f509e104323dce56a1edb7d7c00880bd871ee243834fcc0b27c266106fb8d603"} Jul 02 22:04:22.860637 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:22.860497 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc" Jul 02 22:04:22.860637 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:22.860526 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc" event={"ID":"119616bb-9196-419e-92b7-53ece76e98c4","Type":"ContainerDied","Data":"be22265cd47112dd339f89d9762165d8f62d0cda944ee075a2dccaccd8b290aa"} Jul 02 22:04:22.860637 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:22.860546 2556 scope.go:117] "RemoveContainer" containerID="f509e104323dce56a1edb7d7c00880bd871ee243834fcc0b27c266106fb8d603" Jul 02 22:04:22.869577 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:22.869559 2556 scope.go:117] "RemoveContainer" containerID="f509e104323dce56a1edb7d7c00880bd871ee243834fcc0b27c266106fb8d603" Jul 02 22:04:22.869828 ip-10-0-140-146 kubenswrapper[2556]: E0702 22:04:22.869810 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f509e104323dce56a1edb7d7c00880bd871ee243834fcc0b27c266106fb8d603\": container with ID starting with f509e104323dce56a1edb7d7c00880bd871ee243834fcc0b27c266106fb8d603 not found: ID does not exist" containerID="f509e104323dce56a1edb7d7c00880bd871ee243834fcc0b27c266106fb8d603" Jul 02 22:04:22.869912 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:22.869835 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f509e104323dce56a1edb7d7c00880bd871ee243834fcc0b27c266106fb8d603"} err="failed to get container status \"f509e104323dce56a1edb7d7c00880bd871ee243834fcc0b27c266106fb8d603\": rpc error: code = NotFound desc = could not find container \"f509e104323dce56a1edb7d7c00880bd871ee243834fcc0b27c266106fb8d603\": container with ID starting with f509e104323dce56a1edb7d7c00880bd871ee243834fcc0b27c266106fb8d603 not found: ID does not exist" Jul 02 22:04:22.885537 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:22.885510 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc"] Jul 02 22:04:22.889409 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:22.889389 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-1861b388-569d9f66f6-rtfzc"] Jul 02 22:04:23.815047 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:23.815018 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="119616bb-9196-419e-92b7-53ece76e98c4" path="/var/lib/kubelet/pods/119616bb-9196-419e-92b7-53ece76e98c4/volumes" Jul 02 22:04:24.855631 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:24.855596 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:04:28.027030 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:28.026995 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:28.123714 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:28.123674 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:28.349758 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:28.349673 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:28.594429 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:28.594397 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:28.737323 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:28.737242 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:28.837607 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:28.837571 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:28.958744 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:28.958710 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:34.995741 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:34.995710 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:35.099887 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:35.099834 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:35.198138 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:35.198107 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:35.377032 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:35.377004 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:41.815790 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:41.815760 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:43.214611 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:43.214569 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:48.466859 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:48.466826 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:48.895170 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:48.895136 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:49.086890 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:49.086835 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:51.994149 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:51.994119 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:52.067895 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:52.067851 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs"] Jul 02 22:04:52.068204 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:52.068187 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="119616bb-9196-419e-92b7-53ece76e98c4" containerName="maas-api" Jul 02 22:04:52.068287 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:52.068207 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="119616bb-9196-419e-92b7-53ece76e98c4" containerName="maas-api" Jul 02 22:04:52.068287 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:52.068267 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="119616bb-9196-419e-92b7-53ece76e98c4" containerName="maas-api" Jul 02 22:04:52.070988 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:52.070967 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs" Jul 02 22:04:52.073623 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:52.073605 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-a72fe752-serving-cert\"" Jul 02 22:04:52.073726 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:52.073647 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-66gqh\"" Jul 02 22:04:52.081112 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:52.081091 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs"] Jul 02 22:04:52.092099 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:52.092074 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:52.177349 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:52.177323 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31-openshift-service-ca\") pod \"maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs\" (UID: \"a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31\") " pod="opendatahub/maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs" Jul 02 22:04:52.177496 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:52.177398 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31-maas-api-tls\") pod \"maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs\" (UID: \"a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31\") " pod="opendatahub/maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs" Jul 02 22:04:52.177496 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:52.177425 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfj8n\" (UniqueName: \"kubernetes.io/projected/a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31-kube-api-access-tfj8n\") pod \"maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs\" (UID: \"a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31\") " pod="opendatahub/maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs" Jul 02 22:04:52.183211 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:52.183166 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:52.277944 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:52.277922 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31-openshift-service-ca\") pod \"maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs\" (UID: \"a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31\") " pod="opendatahub/maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs" Jul 02 22:04:52.278054 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:52.277979 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31-maas-api-tls\") pod \"maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs\" (UID: \"a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31\") " pod="opendatahub/maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs" Jul 02 22:04:52.278054 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:52.278003 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfj8n\" (UniqueName: \"kubernetes.io/projected/a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31-kube-api-access-tfj8n\") pod \"maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs\" (UID: \"a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31\") " pod="opendatahub/maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs" Jul 02 22:04:52.278169 ip-10-0-140-146 kubenswrapper[2556]: E0702 22:04:52.278075 2556 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-a72fe752-serving-cert: secret "maas-api-e2e-mt-a72fe752-serving-cert" not found Jul 02 22:04:52.278169 ip-10-0-140-146 kubenswrapper[2556]: E0702 22:04:52.278139 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31-maas-api-tls podName:a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31 nodeName:}" failed. No retries permitted until 2026-07-02 22:04:52.778123987 +0000 UTC m=+1729.497473506 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31-maas-api-tls") pod "maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs" (UID: "a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31") : secret "maas-api-e2e-mt-a72fe752-serving-cert" not found Jul 02 22:04:52.278580 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:52.278558 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31-openshift-service-ca\") pod \"maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs\" (UID: \"a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31\") " pod="opendatahub/maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs" Jul 02 22:04:52.291362 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:52.291332 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfj8n\" (UniqueName: \"kubernetes.io/projected/a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31-kube-api-access-tfj8n\") pod \"maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs\" (UID: \"a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31\") " pod="opendatahub/maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs" Jul 02 22:04:52.577122 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:52.577038 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:52.678362 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:52.678332 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:52.777972 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:52.777942 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:52.781012 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:52.780994 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31-maas-api-tls\") pod \"maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs\" (UID: \"a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31\") " pod="opendatahub/maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs" Jul 02 22:04:52.783531 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:52.783509 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31-maas-api-tls\") pod \"maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs\" (UID: \"a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31\") " pod="opendatahub/maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs" Jul 02 22:04:52.980882 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:52.980785 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs" Jul 02 22:04:53.113779 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:53.113527 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs"] Jul 02 22:04:53.115900 ip-10-0-140-146 kubenswrapper[2556]: W0702 22:04:53.115858 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda90a2e7d_4eb6_413e_8a87_ee7d88f9fe31.slice/crio-4f86693022048242c54bbb8c542b42a627506a86902f6faff79118489cd3b4d5 WatchSource:0}: Error finding container 4f86693022048242c54bbb8c542b42a627506a86902f6faff79118489cd3b4d5: Status 404 returned error can't find the container with id 4f86693022048242c54bbb8c542b42a627506a86902f6faff79118489cd3b4d5 Jul 02 22:04:53.985938 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:53.985839 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs" event={"ID":"a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31","Type":"ContainerStarted","Data":"41836ee4dd1f4d77fe23a150cc20836d2f35c51df26bbcc67c7e6586ba9e1742"} Jul 02 22:04:53.985938 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:53.985902 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs" event={"ID":"a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31","Type":"ContainerStarted","Data":"4f86693022048242c54bbb8c542b42a627506a86902f6faff79118489cd3b4d5"} Jul 02 22:04:53.986257 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:53.985986 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs" Jul 02 22:04:54.006836 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:54.006780 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs" podStartSLOduration=1.5849982200000001 podStartE2EDuration="2.006765898s" totalImagesPullingTime="421.767678ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:04:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:04:54.005805838 +0000 UTC m=+1730.725155379" watchObservedRunningTime="2026-07-02 22:04:54.006765898 +0000 UTC m=+1730.726115439" Jul 02 22:04:56.517352 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:56.517320 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:04:58.540517 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:58.540485 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:04:59.996118 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:04:59.996089 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs" Jul 02 22:05:05.591754 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:05:05.591721 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:05.691140 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:05:05.691110 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:09.899709 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:05:09.899675 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:09.999670 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:05:09.999642 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:11.095639 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:05:11.095606 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:13.288240 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:05:13.288210 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:13.388947 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:05:13.388910 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:14.942988 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:05:14.942952 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:05:27.140250 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:05:27.140213 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:05:29.138685 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:05:29.138650 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:29.661592 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:05:29.661566 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs"] Jul 02 22:05:29.661818 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:05:29.661797 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs" podUID="a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31" containerName="maas-api" containerID="cri-o://41836ee4dd1f4d77fe23a150cc20836d2f35c51df26bbcc67c7e6586ba9e1742" gracePeriod=30 Jul 02 22:05:29.693362 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:05:29.693337 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:29.910013 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:05:29.909992 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs" Jul 02 22:05:29.976579 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:05:29.976508 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31-maas-api-tls\") pod \"a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31\" (UID: \"a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31\") " Jul 02 22:05:29.976718 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:05:29.976582 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31-openshift-service-ca\") pod \"a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31\" (UID: \"a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31\") " Jul 02 22:05:29.976718 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:05:29.976618 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tfj8n\" (UniqueName: \"kubernetes.io/projected/a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31-kube-api-access-tfj8n\") pod \"a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31\" (UID: \"a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31\") " Jul 02 22:05:29.976967 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:05:29.976935 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31" (UID: "a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:05:29.978819 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:05:29.978790 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31" (UID: "a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 22:05:29.978819 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:05:29.978799 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31-kube-api-access-tfj8n" (OuterVolumeSpecName: "kube-api-access-tfj8n") pod "a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31" (UID: "a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31"). InnerVolumeSpecName "kube-api-access-tfj8n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:05:30.077478 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:05:30.077443 2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31-openshift-service-ca\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:05:30.077478 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:05:30.077472 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tfj8n\" (UniqueName: \"kubernetes.io/projected/a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31-kube-api-access-tfj8n\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:05:30.077478 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:05:30.077482 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31-maas-api-tls\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:05:30.125217 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:05:30.125190 2556 generic.go:356] "Generic (PLEG): container finished" podID="a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31" containerID="41836ee4dd1f4d77fe23a150cc20836d2f35c51df26bbcc67c7e6586ba9e1742" exitCode=0 Jul 02 22:05:30.125364 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:05:30.125231 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs" event={"ID":"a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31","Type":"ContainerDied","Data":"41836ee4dd1f4d77fe23a150cc20836d2f35c51df26bbcc67c7e6586ba9e1742"} Jul 02 22:05:30.125364 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:05:30.125238 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs" Jul 02 22:05:30.125364 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:05:30.125274 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs" event={"ID":"a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31","Type":"ContainerDied","Data":"4f86693022048242c54bbb8c542b42a627506a86902f6faff79118489cd3b4d5"} Jul 02 22:05:30.125364 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:05:30.125297 2556 scope.go:117] "RemoveContainer" containerID="41836ee4dd1f4d77fe23a150cc20836d2f35c51df26bbcc67c7e6586ba9e1742" Jul 02 22:05:30.133917 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:05:30.133895 2556 scope.go:117] "RemoveContainer" containerID="41836ee4dd1f4d77fe23a150cc20836d2f35c51df26bbcc67c7e6586ba9e1742" Jul 02 22:05:30.134137 ip-10-0-140-146 kubenswrapper[2556]: E0702 22:05:30.134119 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41836ee4dd1f4d77fe23a150cc20836d2f35c51df26bbcc67c7e6586ba9e1742\": container with ID starting with 41836ee4dd1f4d77fe23a150cc20836d2f35c51df26bbcc67c7e6586ba9e1742 not found: ID does not exist" containerID="41836ee4dd1f4d77fe23a150cc20836d2f35c51df26bbcc67c7e6586ba9e1742" Jul 02 22:05:30.134192 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:05:30.134146 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41836ee4dd1f4d77fe23a150cc20836d2f35c51df26bbcc67c7e6586ba9e1742"} err="failed to get container status \"41836ee4dd1f4d77fe23a150cc20836d2f35c51df26bbcc67c7e6586ba9e1742\": rpc error: code = NotFound desc = could not find container \"41836ee4dd1f4d77fe23a150cc20836d2f35c51df26bbcc67c7e6586ba9e1742\": container with ID starting with 41836ee4dd1f4d77fe23a150cc20836d2f35c51df26bbcc67c7e6586ba9e1742 not found: ID does not exist" Jul 02 22:05:30.147525 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:05:30.147501 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs"] Jul 02 22:05:30.154965 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:05:30.154936 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-a72fe752-789cfb9bfb-4kwgs"] Jul 02 22:05:31.815286 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:05:31.815257 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31" path="/var/lib/kubelet/pods/a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31/volumes" Jul 02 22:05:35.171801 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:05:35.171764 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:40.250473 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:05:40.250440 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:05:40.639497 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:05:40.639465 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:06:03.790951 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:06:03.790922 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8jx8_7a32947e-1a4e-4138-8367-6def69748311/ovn-acl-logging/0.log" Jul 02 22:06:03.792544 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:06:03.792526 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8jx8_7a32947e-1a4e-4138-8367-6def69748311/ovn-acl-logging/0.log" Jul 02 22:06:16.663986 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:06:16.663953 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:16.898648 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:06:16.898619 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:17.601320 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:06:17.601286 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:19.938590 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:06:19.938558 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:06:23.535373 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:06:23.535335 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:46.865436 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:06:46.865396 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:46.940981 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:06:46.940956 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:47.148228 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:06:47.148146 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:47.380154 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:06:47.380115 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:48.441836 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:06:48.441793 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:48.540287 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:06:48.540246 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:48.639631 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:06:48.639605 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:49.356188 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:06:49.356156 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:49.454070 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:06:49.454035 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:49.553662 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:06:49.553635 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:50.925893 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:06:50.925843 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:50.952781 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:06:50.952755 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:06:57.003392 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:06:57.003350 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:57.103910 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:06:57.103856 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:06:57.670641 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:06:57.670608 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:07:03.656271 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:03.656237 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:07:03.759196 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:03.759160 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:07:03.856386 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:03.856356 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:07:05.143396 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:05.143369 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:07:14.436540 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:14.436508 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:07:14.743025 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:14.742949 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:07:35.704609 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:35.704526 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:07:41.670974 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:41.670935 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:07:41.770712 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:41.770681 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:07:42.040169 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:42.040138 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:07:42.375717 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:42.375642 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:07:42.612407 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:42.612379 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:07:46.006960 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:46.006922 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:07:46.052112 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:46.052079 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-624219-58b7f44cb7-xmtxq"] Jul 02 22:07:46.052415 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:46.052399 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31" containerName="maas-api" Jul 02 22:07:46.052501 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:46.052419 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31" containerName="maas-api" Jul 02 22:07:46.052501 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:46.052485 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="a90a2e7d-4eb6-413e-8a87-ee7d88f9fe31" containerName="maas-api" Jul 02 22:07:46.055102 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:46.055083 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-624219-58b7f44cb7-xmtxq" Jul 02 22:07:46.060109 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:46.060036 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-66gqh\"" Jul 02 22:07:46.060109 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:46.060065 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-624219-serving-cert\"" Jul 02 22:07:46.070557 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:46.070533 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-624219-58b7f44cb7-xmtxq"] Jul 02 22:07:46.166991 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:46.166963 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xntfv\" (UniqueName: \"kubernetes.io/projected/e137ddd0-010e-450e-bfee-2ed20e645259-kube-api-access-xntfv\") pod \"maas-api-e2e-inf-a-624219-58b7f44cb7-xmtxq\" (UID: \"e137ddd0-010e-450e-bfee-2ed20e645259\") " pod="opendatahub/maas-api-e2e-inf-a-624219-58b7f44cb7-xmtxq" Jul 02 22:07:46.167129 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:46.167032 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e137ddd0-010e-450e-bfee-2ed20e645259-openshift-service-ca\") pod \"maas-api-e2e-inf-a-624219-58b7f44cb7-xmtxq\" (UID: \"e137ddd0-010e-450e-bfee-2ed20e645259\") " pod="opendatahub/maas-api-e2e-inf-a-624219-58b7f44cb7-xmtxq" Jul 02 22:07:46.167129 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:46.167061 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e137ddd0-010e-450e-bfee-2ed20e645259-maas-api-tls\") pod \"maas-api-e2e-inf-a-624219-58b7f44cb7-xmtxq\" (UID: \"e137ddd0-010e-450e-bfee-2ed20e645259\") " pod="opendatahub/maas-api-e2e-inf-a-624219-58b7f44cb7-xmtxq" Jul 02 22:07:46.168091 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:46.168068 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:07:46.267693 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:46.267623 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xntfv\" (UniqueName: \"kubernetes.io/projected/e137ddd0-010e-450e-bfee-2ed20e645259-kube-api-access-xntfv\") pod \"maas-api-e2e-inf-a-624219-58b7f44cb7-xmtxq\" (UID: \"e137ddd0-010e-450e-bfee-2ed20e645259\") " pod="opendatahub/maas-api-e2e-inf-a-624219-58b7f44cb7-xmtxq" Jul 02 22:07:46.267693 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:46.267670 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e137ddd0-010e-450e-bfee-2ed20e645259-openshift-service-ca\") pod \"maas-api-e2e-inf-a-624219-58b7f44cb7-xmtxq\" (UID: \"e137ddd0-010e-450e-bfee-2ed20e645259\") " pod="opendatahub/maas-api-e2e-inf-a-624219-58b7f44cb7-xmtxq" Jul 02 22:07:46.267693 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:46.267691 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e137ddd0-010e-450e-bfee-2ed20e645259-maas-api-tls\") pod \"maas-api-e2e-inf-a-624219-58b7f44cb7-xmtxq\" (UID: \"e137ddd0-010e-450e-bfee-2ed20e645259\") " pod="opendatahub/maas-api-e2e-inf-a-624219-58b7f44cb7-xmtxq" Jul 02 22:07:46.268279 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:46.268253 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e137ddd0-010e-450e-bfee-2ed20e645259-openshift-service-ca\") pod \"maas-api-e2e-inf-a-624219-58b7f44cb7-xmtxq\" (UID: \"e137ddd0-010e-450e-bfee-2ed20e645259\") " pod="opendatahub/maas-api-e2e-inf-a-624219-58b7f44cb7-xmtxq" Jul 02 22:07:46.270262 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:46.270239 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e137ddd0-010e-450e-bfee-2ed20e645259-maas-api-tls\") pod \"maas-api-e2e-inf-a-624219-58b7f44cb7-xmtxq\" (UID: \"e137ddd0-010e-450e-bfee-2ed20e645259\") " pod="opendatahub/maas-api-e2e-inf-a-624219-58b7f44cb7-xmtxq" Jul 02 22:07:46.294566 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:46.294539 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xntfv\" (UniqueName: \"kubernetes.io/projected/e137ddd0-010e-450e-bfee-2ed20e645259-kube-api-access-xntfv\") pod \"maas-api-e2e-inf-a-624219-58b7f44cb7-xmtxq\" (UID: \"e137ddd0-010e-450e-bfee-2ed20e645259\") " pod="opendatahub/maas-api-e2e-inf-a-624219-58b7f44cb7-xmtxq" Jul 02 22:07:46.331076 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:46.331050 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:07:46.366987 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:46.366967 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-624219-58b7f44cb7-xmtxq" Jul 02 22:07:46.503568 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:46.503535 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-624219-58b7f44cb7-xmtxq"] Jul 02 22:07:46.506754 ip-10-0-140-146 kubenswrapper[2556]: W0702 22:07:46.506728 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode137ddd0_010e_450e_bfee_2ed20e645259.slice/crio-23f180e54bfc1598d25c3fe47e4cb16fb1e6468db4181df93e82701b9cdd1a55 WatchSource:0}: Error finding container 23f180e54bfc1598d25c3fe47e4cb16fb1e6468db4181df93e82701b9cdd1a55: Status 404 returned error can't find the container with id 23f180e54bfc1598d25c3fe47e4cb16fb1e6468db4181df93e82701b9cdd1a55 Jul 02 22:07:46.508465 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:46.508448 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 22:07:46.624805 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:46.624728 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-624219-58b7f44cb7-xmtxq" event={"ID":"e137ddd0-010e-450e-bfee-2ed20e645259","Type":"ContainerStarted","Data":"23f180e54bfc1598d25c3fe47e4cb16fb1e6468db4181df93e82701b9cdd1a55"} Jul 02 22:07:46.800753 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:46.800724 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:07:46.919910 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:46.919792 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:07:47.630295 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:47.630265 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-624219-58b7f44cb7-xmtxq" event={"ID":"e137ddd0-010e-450e-bfee-2ed20e645259","Type":"ContainerStarted","Data":"47eead8f543032c1ba6a220869f937bc83cc7e4a869ba19acd0152b625309b34"} Jul 02 22:07:47.630654 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:47.630389 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-624219-58b7f44cb7-xmtxq" Jul 02 22:07:47.654177 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:47.654138 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-624219-58b7f44cb7-xmtxq" podStartSLOduration=1.209035022 podStartE2EDuration="1.654124344s" totalImagesPullingTime="445.089322ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:07:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:07:47.65314089 +0000 UTC m=+1904.372490432" watchObservedRunningTime="2026-07-02 22:07:47.654124344 +0000 UTC m=+1904.373473885" Jul 02 22:07:50.399803 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:50.399765 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:07:50.757961 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:50.757931 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:07:51.053641 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:51.053562 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr"] Jul 02 22:07:51.056829 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:51.056806 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:07:51.059347 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:51.059327 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-d372c0-openshift-default-dockercfg-tz7gn\"" Jul 02 22:07:51.072937 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:51.072912 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr"] Jul 02 22:07:51.102050 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:51.102020 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:07:51.208501 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:51.208467 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c73068b3-2dd8-4c8c-bc74-5860f03aea13-istio-envoy\") pod \"e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr\" (UID: \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\") " pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:07:51.208668 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:51.208505 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c73068b3-2dd8-4c8c-bc74-5860f03aea13-istio-podinfo\") pod \"e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr\" (UID: \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\") " pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:07:51.208668 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:51.208544 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c73068b3-2dd8-4c8c-bc74-5860f03aea13-istiod-ca-cert\") pod \"e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr\" (UID: \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\") " pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:07:51.208668 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:51.208617 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c73068b3-2dd8-4c8c-bc74-5860f03aea13-workload-certs\") pod \"e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr\" (UID: \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\") " pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:07:51.208668 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:51.208649 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c73068b3-2dd8-4c8c-bc74-5860f03aea13-workload-socket\") pod \"e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr\" (UID: \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\") " pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:07:51.208800 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:51.208672 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgm92\" (UniqueName: \"kubernetes.io/projected/c73068b3-2dd8-4c8c-bc74-5860f03aea13-kube-api-access-fgm92\") pod \"e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr\" (UID: \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\") " pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:07:51.208800 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:51.208700 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c73068b3-2dd8-4c8c-bc74-5860f03aea13-credential-socket\") pod \"e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr\" (UID: \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\") " pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:07:51.208800 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:51.208732 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c73068b3-2dd8-4c8c-bc74-5860f03aea13-istio-token\") pod \"e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr\" (UID: \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\") " pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:07:51.208800 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:51.208756 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c73068b3-2dd8-4c8c-bc74-5860f03aea13-istio-data\") pod \"e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr\" (UID: \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\") " pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:07:51.309581 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:51.309510 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c73068b3-2dd8-4c8c-bc74-5860f03aea13-workload-socket\") pod \"e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr\" (UID: \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\") " pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:07:51.309581 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:51.309542 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgm92\" (UniqueName: \"kubernetes.io/projected/c73068b3-2dd8-4c8c-bc74-5860f03aea13-kube-api-access-fgm92\") pod \"e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr\" (UID: \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\") " pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:07:51.309581 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:51.309566 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c73068b3-2dd8-4c8c-bc74-5860f03aea13-credential-socket\") pod \"e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr\" (UID: \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\") " pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:07:51.309855 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:51.309602 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c73068b3-2dd8-4c8c-bc74-5860f03aea13-istio-token\") pod \"e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr\" (UID: \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\") " pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:07:51.309855 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:51.309627 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c73068b3-2dd8-4c8c-bc74-5860f03aea13-istio-data\") pod \"e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr\" (UID: \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\") " pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:07:51.309855 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:51.309808 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c73068b3-2dd8-4c8c-bc74-5860f03aea13-istio-envoy\") pod \"e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr\" (UID: \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\") " pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:07:51.310041 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:51.309848 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c73068b3-2dd8-4c8c-bc74-5860f03aea13-istio-podinfo\") pod \"e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr\" (UID: \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\") " pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:07:51.310041 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:51.309939 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c73068b3-2dd8-4c8c-bc74-5860f03aea13-istiod-ca-cert\") pod \"e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr\" (UID: \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\") " pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:07:51.310041 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:51.309987 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c73068b3-2dd8-4c8c-bc74-5860f03aea13-workload-certs\") pod \"e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr\" (UID: \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\") " pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:07:51.310041 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:51.309992 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c73068b3-2dd8-4c8c-bc74-5860f03aea13-workload-socket\") pod \"e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr\" (UID: \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\") " pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:07:51.310214 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:51.310039 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c73068b3-2dd8-4c8c-bc74-5860f03aea13-credential-socket\") pod \"e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr\" (UID: \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\") " pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:07:51.310214 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:51.310109 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c73068b3-2dd8-4c8c-bc74-5860f03aea13-istio-data\") pod \"e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr\" (UID: \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\") " pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:07:51.310302 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:51.310239 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c73068b3-2dd8-4c8c-bc74-5860f03aea13-workload-certs\") pod \"e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr\" (UID: \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\") " pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:07:51.310545 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:51.310527 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c73068b3-2dd8-4c8c-bc74-5860f03aea13-istiod-ca-cert\") pod \"e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr\" (UID: \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\") " pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:07:51.312123 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:51.312106 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c73068b3-2dd8-4c8c-bc74-5860f03aea13-istio-envoy\") pod \"e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr\" (UID: \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\") " pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:07:51.312338 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:51.312321 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c73068b3-2dd8-4c8c-bc74-5860f03aea13-istio-podinfo\") pod \"e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr\" (UID: \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\") " pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:07:51.327476 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:51.327454 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c73068b3-2dd8-4c8c-bc74-5860f03aea13-istio-token\") pod \"e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr\" (UID: \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\") " pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:07:51.328636 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:51.328615 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgm92\" (UniqueName: \"kubernetes.io/projected/c73068b3-2dd8-4c8c-bc74-5860f03aea13-kube-api-access-fgm92\") pod \"e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr\" (UID: \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\") " pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:07:51.344734 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:51.344710 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:07:51.366724 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:51.366703 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:07:51.502757 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:51.502735 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr"] Jul 02 22:07:51.505087 ip-10-0-140-146 kubenswrapper[2556]: W0702 22:07:51.505059 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc73068b3_2dd8_4c8c_bc74_5860f03aea13.slice/crio-511d3718742811a02985c6fe247cbe3aeb299df58dd62dce8693fe122477b9be WatchSource:0}: Error finding container 511d3718742811a02985c6fe247cbe3aeb299df58dd62dce8693fe122477b9be: Status 404 returned error can't find the container with id 511d3718742811a02985c6fe247cbe3aeb299df58dd62dce8693fe122477b9be Jul 02 22:07:51.646624 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:51.646542 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" event={"ID":"c73068b3-2dd8-4c8c-bc74-5860f03aea13","Type":"ContainerStarted","Data":"511d3718742811a02985c6fe247cbe3aeb299df58dd62dce8693fe122477b9be"} Jul 02 22:07:53.538578 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:53.538544 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:07:53.639739 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:53.639715 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-624219-58b7f44cb7-xmtxq" Jul 02 22:07:55.495097 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:55.495064 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 22:07:55.495467 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:55.495149 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 22:07:55.495467 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:55.495196 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 22:07:55.664569 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:55.664534 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" event={"ID":"c73068b3-2dd8-4c8c-bc74-5860f03aea13","Type":"ContainerStarted","Data":"66d3a60c4e96bdf908335bde323a66300b3fc09c96f7870dc0561b5e0e244cce"} Jul 02 22:07:55.689600 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:55.689528 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" podStartSLOduration=0.701812784 podStartE2EDuration="4.689494951s" totalImagesPullingTime="3.987682167s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:07:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:07:55.688826161 +0000 UTC m=+1912.408175695" watchObservedRunningTime="2026-07-02 22:07:55.689494951 +0000 UTC m=+1912.408844500" Jul 02 22:07:56.367445 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:56.367410 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:07:56.372401 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:56.372372 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:07:56.669325 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:56.669244 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:07:56.670143 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:56.670123 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:07:57.684517 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:57.684489 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:07:57.784334 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:57.784302 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:07:57.885386 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:57.885358 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-d372c0-7d6f54c987-tqd6v"] Jul 02 22:07:57.888968 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:57.888943 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-d372c0-7d6f54c987-tqd6v" Jul 02 22:07:57.891516 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:57.891494 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-d372c0-serving-cert\"" Jul 02 22:07:57.899861 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:57.899840 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-d372c0-7d6f54c987-tqd6v"] Jul 02 22:07:57.939615 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:57.939544 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:07:58.067044 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:58.067008 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzv9q\" (UniqueName: \"kubernetes.io/projected/67f04a11-77de-4d3f-abcf-b0b0ce617a9e-kube-api-access-xzv9q\") pod \"maas-api-e2e-inf-b-d372c0-7d6f54c987-tqd6v\" (UID: \"67f04a11-77de-4d3f-abcf-b0b0ce617a9e\") " pod="opendatahub/maas-api-e2e-inf-b-d372c0-7d6f54c987-tqd6v" Jul 02 22:07:58.067208 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:58.067066 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67f04a11-77de-4d3f-abcf-b0b0ce617a9e-maas-api-tls\") pod \"maas-api-e2e-inf-b-d372c0-7d6f54c987-tqd6v\" (UID: \"67f04a11-77de-4d3f-abcf-b0b0ce617a9e\") " pod="opendatahub/maas-api-e2e-inf-b-d372c0-7d6f54c987-tqd6v" Jul 02 22:07:58.067208 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:58.067119 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67f04a11-77de-4d3f-abcf-b0b0ce617a9e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-d372c0-7d6f54c987-tqd6v\" (UID: \"67f04a11-77de-4d3f-abcf-b0b0ce617a9e\") " pod="opendatahub/maas-api-e2e-inf-b-d372c0-7d6f54c987-tqd6v" Jul 02 22:07:58.167660 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:58.167633 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67f04a11-77de-4d3f-abcf-b0b0ce617a9e-maas-api-tls\") pod \"maas-api-e2e-inf-b-d372c0-7d6f54c987-tqd6v\" (UID: \"67f04a11-77de-4d3f-abcf-b0b0ce617a9e\") " pod="opendatahub/maas-api-e2e-inf-b-d372c0-7d6f54c987-tqd6v" Jul 02 22:07:58.167798 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:58.167668 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67f04a11-77de-4d3f-abcf-b0b0ce617a9e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-d372c0-7d6f54c987-tqd6v\" (UID: \"67f04a11-77de-4d3f-abcf-b0b0ce617a9e\") " pod="opendatahub/maas-api-e2e-inf-b-d372c0-7d6f54c987-tqd6v" Jul 02 22:07:58.167798 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:58.167717 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzv9q\" (UniqueName: \"kubernetes.io/projected/67f04a11-77de-4d3f-abcf-b0b0ce617a9e-kube-api-access-xzv9q\") pod \"maas-api-e2e-inf-b-d372c0-7d6f54c987-tqd6v\" (UID: \"67f04a11-77de-4d3f-abcf-b0b0ce617a9e\") " pod="opendatahub/maas-api-e2e-inf-b-d372c0-7d6f54c987-tqd6v" Jul 02 22:07:58.168380 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:58.168357 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67f04a11-77de-4d3f-abcf-b0b0ce617a9e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-d372c0-7d6f54c987-tqd6v\" (UID: \"67f04a11-77de-4d3f-abcf-b0b0ce617a9e\") " pod="opendatahub/maas-api-e2e-inf-b-d372c0-7d6f54c987-tqd6v" Jul 02 22:07:58.170349 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:58.170328 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67f04a11-77de-4d3f-abcf-b0b0ce617a9e-maas-api-tls\") pod \"maas-api-e2e-inf-b-d372c0-7d6f54c987-tqd6v\" (UID: \"67f04a11-77de-4d3f-abcf-b0b0ce617a9e\") " pod="opendatahub/maas-api-e2e-inf-b-d372c0-7d6f54c987-tqd6v" Jul 02 22:07:58.171690 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:58.171667 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:07:58.176893 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:58.176853 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzv9q\" (UniqueName: \"kubernetes.io/projected/67f04a11-77de-4d3f-abcf-b0b0ce617a9e-kube-api-access-xzv9q\") pod \"maas-api-e2e-inf-b-d372c0-7d6f54c987-tqd6v\" (UID: \"67f04a11-77de-4d3f-abcf-b0b0ce617a9e\") " pod="opendatahub/maas-api-e2e-inf-b-d372c0-7d6f54c987-tqd6v" Jul 02 22:07:58.199716 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:58.199664 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-d372c0-7d6f54c987-tqd6v" Jul 02 22:07:58.325587 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:58.325552 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-d372c0-7d6f54c987-tqd6v"] Jul 02 22:07:58.327643 ip-10-0-140-146 kubenswrapper[2556]: W0702 22:07:58.327616 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67f04a11_77de_4d3f_abcf_b0b0ce617a9e.slice/crio-231442abc6b8d13cc10d55afe1c000ad4edb61d0b358caefeff68cf48d2886d4 WatchSource:0}: Error finding container 231442abc6b8d13cc10d55afe1c000ad4edb61d0b358caefeff68cf48d2886d4: Status 404 returned error can't find the container with id 231442abc6b8d13cc10d55afe1c000ad4edb61d0b358caefeff68cf48d2886d4 Jul 02 22:07:58.433445 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:58.433419 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:07:58.542105 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:58.542076 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:07:58.677756 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:58.677717 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-d372c0-7d6f54c987-tqd6v" event={"ID":"67f04a11-77de-4d3f-abcf-b0b0ce617a9e","Type":"ContainerStarted","Data":"231442abc6b8d13cc10d55afe1c000ad4edb61d0b358caefeff68cf48d2886d4"} Jul 02 22:07:59.683827 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:59.683793 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-d372c0-7d6f54c987-tqd6v" event={"ID":"67f04a11-77de-4d3f-abcf-b0b0ce617a9e","Type":"ContainerStarted","Data":"5288c05dc2b7e0ddd85fa92e8523fe8553a58aba9209807b7f055b931f31f67b"} Jul 02 22:07:59.684223 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:59.683927 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-d372c0-7d6f54c987-tqd6v" Jul 02 22:07:59.702574 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:07:59.702532 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-d372c0-7d6f54c987-tqd6v" podStartSLOduration=2.210484493 podStartE2EDuration="2.702516535s" totalImagesPullingTime="492.032042ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:07:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:07:59.701272211 +0000 UTC m=+1916.420621752" watchObservedRunningTime="2026-07-02 22:07:59.702516535 +0000 UTC m=+1916.421866076" Jul 02 22:08:02.208171 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:02.208137 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:08:05.700085 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:05.700054 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-d372c0-7d6f54c987-tqd6v" Jul 02 22:08:07.540071 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:07.540023 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:08:07.669924 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:07.669860 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:08:07.945743 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:07.945655 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:08:09.728031 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:09.727926 2556 generic.go:356] "Generic (PLEG): container finished" podID="c73068b3-2dd8-4c8c-bc74-5860f03aea13" containerID="66d3a60c4e96bdf908335bde323a66300b3fc09c96f7870dc0561b5e0e244cce" exitCode=0 Jul 02 22:08:09.728031 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:09.727992 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" event={"ID":"c73068b3-2dd8-4c8c-bc74-5860f03aea13","Type":"ContainerDied","Data":"66d3a60c4e96bdf908335bde323a66300b3fc09c96f7870dc0561b5e0e244cce"} Jul 02 22:08:09.728578 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:09.728435 2556 scope.go:117] "RemoveContainer" containerID="66d3a60c4e96bdf908335bde323a66300b3fc09c96f7870dc0561b5e0e244cce" Jul 02 22:08:09.729449 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:09.729420 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 22:08:09.729508 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:09.729488 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 22:08:09.729549 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:09.729520 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 22:08:10.733510 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:10.733476 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" event={"ID":"c73068b3-2dd8-4c8c-bc74-5860f03aea13","Type":"ContainerStarted","Data":"c63b447889376da359a95e945201a922d915a31ebeafd3c23e9ef0854d0d34f3"} Jul 02 22:08:11.056111 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:11.056066 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:08:11.367345 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:11.367263 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:08:11.368374 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:11.368348 2556 patch_prober.go:28] interesting pod/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body= Jul 02 22:08:11.368493 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:11.368410 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" podUID="c73068b3-2dd8-4c8c-bc74-5860f03aea13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" Jul 02 22:08:11.669532 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:11.669449 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:08:12.367601 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:12.367561 2556 patch_prober.go:28] interesting pod/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body= Jul 02 22:08:12.368019 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:12.367616 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" podUID="c73068b3-2dd8-4c8c-bc74-5860f03aea13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" Jul 02 22:08:13.367223 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:13.367176 2556 patch_prober.go:28] interesting pod/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body= Jul 02 22:08:13.367400 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:13.367242 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" podUID="c73068b3-2dd8-4c8c-bc74-5860f03aea13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" Jul 02 22:08:14.367282 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:14.367244 2556 patch_prober.go:28] interesting pod/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body= Jul 02 22:08:14.367647 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:14.367296 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" podUID="c73068b3-2dd8-4c8c-bc74-5860f03aea13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" Jul 02 22:08:15.372218 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:15.372188 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:08:15.373110 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:15.373095 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:08:16.737687 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:16.737654 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:08:19.119917 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:19.119853 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:08:19.217092 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:19.217047 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:08:35.154643 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:35.154606 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:08:39.425424 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:39.425382 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:08:53.532661 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:53.532627 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:08:55.570079 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:55.569980 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:08:56.490534 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:56.490457 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:08:56.663177 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:56.663108 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:08:57.055721 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:57.055686 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:08:57.158205 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:57.158172 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:08:57.257759 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:57.257727 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:08:57.295340 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:57.295309 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-624219-58b7f44cb7-xmtxq"] Jul 02 22:08:57.295523 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:57.295496 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-624219-58b7f44cb7-xmtxq" podUID="e137ddd0-010e-450e-bfee-2ed20e645259" containerName="maas-api" containerID="cri-o://47eead8f543032c1ba6a220869f937bc83cc7e4a869ba19acd0152b625309b34" gracePeriod=30 Jul 02 22:08:57.371469 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:57.371439 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:08:57.540532 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:57.540511 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-624219-58b7f44cb7-xmtxq" Jul 02 22:08:57.634066 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:57.633979 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xntfv\" (UniqueName: \"kubernetes.io/projected/e137ddd0-010e-450e-bfee-2ed20e645259-kube-api-access-xntfv\") pod \"e137ddd0-010e-450e-bfee-2ed20e645259\" (UID: \"e137ddd0-010e-450e-bfee-2ed20e645259\") " Jul 02 22:08:57.634066 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:57.634070 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e137ddd0-010e-450e-bfee-2ed20e645259-maas-api-tls\") pod \"e137ddd0-010e-450e-bfee-2ed20e645259\" (UID: \"e137ddd0-010e-450e-bfee-2ed20e645259\") " Jul 02 22:08:57.634257 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:57.634129 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e137ddd0-010e-450e-bfee-2ed20e645259-openshift-service-ca\") pod \"e137ddd0-010e-450e-bfee-2ed20e645259\" (UID: \"e137ddd0-010e-450e-bfee-2ed20e645259\") " Jul 02 22:08:57.634503 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:57.634478 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e137ddd0-010e-450e-bfee-2ed20e645259-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e137ddd0-010e-450e-bfee-2ed20e645259" (UID: "e137ddd0-010e-450e-bfee-2ed20e645259"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:08:57.636144 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:57.636121 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e137ddd0-010e-450e-bfee-2ed20e645259-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e137ddd0-010e-450e-bfee-2ed20e645259" (UID: "e137ddd0-010e-450e-bfee-2ed20e645259"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 22:08:57.636228 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:57.636145 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e137ddd0-010e-450e-bfee-2ed20e645259-kube-api-access-xntfv" (OuterVolumeSpecName: "kube-api-access-xntfv") pod "e137ddd0-010e-450e-bfee-2ed20e645259" (UID: "e137ddd0-010e-450e-bfee-2ed20e645259"). InnerVolumeSpecName "kube-api-access-xntfv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:08:57.734591 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:57.734563 2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e137ddd0-010e-450e-bfee-2ed20e645259-openshift-service-ca\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:08:57.734591 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:57.734587 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xntfv\" (UniqueName: \"kubernetes.io/projected/e137ddd0-010e-450e-bfee-2ed20e645259-kube-api-access-xntfv\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:08:57.734591 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:57.734596 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e137ddd0-010e-450e-bfee-2ed20e645259-maas-api-tls\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:08:57.915272 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:57.915185 2556 generic.go:356] "Generic (PLEG): container finished" podID="e137ddd0-010e-450e-bfee-2ed20e645259" containerID="47eead8f543032c1ba6a220869f937bc83cc7e4a869ba19acd0152b625309b34" exitCode=0 Jul 02 22:08:57.915272 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:57.915248 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-624219-58b7f44cb7-xmtxq" Jul 02 22:08:57.915457 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:57.915272 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-624219-58b7f44cb7-xmtxq" event={"ID":"e137ddd0-010e-450e-bfee-2ed20e645259","Type":"ContainerDied","Data":"47eead8f543032c1ba6a220869f937bc83cc7e4a869ba19acd0152b625309b34"} Jul 02 22:08:57.915457 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:57.915344 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-624219-58b7f44cb7-xmtxq" event={"ID":"e137ddd0-010e-450e-bfee-2ed20e645259","Type":"ContainerDied","Data":"23f180e54bfc1598d25c3fe47e4cb16fb1e6468db4181df93e82701b9cdd1a55"} Jul 02 22:08:57.915457 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:57.915367 2556 scope.go:117] "RemoveContainer" containerID="47eead8f543032c1ba6a220869f937bc83cc7e4a869ba19acd0152b625309b34" Jul 02 22:08:57.923344 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:57.923323 2556 scope.go:117] "RemoveContainer" containerID="47eead8f543032c1ba6a220869f937bc83cc7e4a869ba19acd0152b625309b34" Jul 02 22:08:57.923559 ip-10-0-140-146 kubenswrapper[2556]: E0702 22:08:57.923539 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47eead8f543032c1ba6a220869f937bc83cc7e4a869ba19acd0152b625309b34\": container with ID starting with 47eead8f543032c1ba6a220869f937bc83cc7e4a869ba19acd0152b625309b34 not found: ID does not exist" containerID="47eead8f543032c1ba6a220869f937bc83cc7e4a869ba19acd0152b625309b34" Jul 02 22:08:57.923600 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:57.923567 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47eead8f543032c1ba6a220869f937bc83cc7e4a869ba19acd0152b625309b34"} err="failed to get container status \"47eead8f543032c1ba6a220869f937bc83cc7e4a869ba19acd0152b625309b34\": rpc error: code = NotFound desc = could not find container \"47eead8f543032c1ba6a220869f937bc83cc7e4a869ba19acd0152b625309b34\": container with ID starting with 47eead8f543032c1ba6a220869f937bc83cc7e4a869ba19acd0152b625309b34 not found: ID does not exist" Jul 02 22:08:57.931144 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:57.931121 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-624219-58b7f44cb7-xmtxq"] Jul 02 22:08:57.937103 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:57.937083 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-624219-58b7f44cb7-xmtxq"] Jul 02 22:08:59.816446 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:08:59.816413 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e137ddd0-010e-450e-bfee-2ed20e645259" path="/var/lib/kubelet/pods/e137ddd0-010e-450e-bfee-2ed20e645259/volumes" Jul 02 22:09:02.126224 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:02.126169 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:10.535291 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:10.535256 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:09:39.401422 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:39.401374 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:39.679076 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:39.678986 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:39.975910 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:39.975799 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:40.101823 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:40.101797 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:40.492836 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:40.492803 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:40.593841 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:40.593810 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:40.694089 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:40.694048 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:40.794223 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:40.794182 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-d372c0-7d6f54c987-tqd6v"] Jul 02 22:09:40.794420 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:40.794394 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-d372c0-7d6f54c987-tqd6v" podUID="67f04a11-77de-4d3f-abcf-b0b0ce617a9e" containerName="maas-api" containerID="cri-o://5288c05dc2b7e0ddd85fa92e8523fe8553a58aba9209807b7f055b931f31f67b" gracePeriod=30 Jul 02 22:09:40.818684 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:40.818657 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:41.036306 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:41.036282 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-d372c0-7d6f54c987-tqd6v" Jul 02 22:09:41.045100 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:41.045059 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67f04a11-77de-4d3f-abcf-b0b0ce617a9e-openshift-service-ca\") pod \"67f04a11-77de-4d3f-abcf-b0b0ce617a9e\" (UID: \"67f04a11-77de-4d3f-abcf-b0b0ce617a9e\") " Jul 02 22:09:41.045100 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:41.045101 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67f04a11-77de-4d3f-abcf-b0b0ce617a9e-maas-api-tls\") pod \"67f04a11-77de-4d3f-abcf-b0b0ce617a9e\" (UID: \"67f04a11-77de-4d3f-abcf-b0b0ce617a9e\") " Jul 02 22:09:41.045221 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:41.045133 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xzv9q\" (UniqueName: \"kubernetes.io/projected/67f04a11-77de-4d3f-abcf-b0b0ce617a9e-kube-api-access-xzv9q\") pod \"67f04a11-77de-4d3f-abcf-b0b0ce617a9e\" (UID: \"67f04a11-77de-4d3f-abcf-b0b0ce617a9e\") " Jul 02 22:09:41.045377 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:41.045356 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67f04a11-77de-4d3f-abcf-b0b0ce617a9e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "67f04a11-77de-4d3f-abcf-b0b0ce617a9e" (UID: "67f04a11-77de-4d3f-abcf-b0b0ce617a9e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:09:41.047296 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:41.047269 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67f04a11-77de-4d3f-abcf-b0b0ce617a9e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "67f04a11-77de-4d3f-abcf-b0b0ce617a9e" (UID: "67f04a11-77de-4d3f-abcf-b0b0ce617a9e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 22:09:41.047296 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:41.047269 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67f04a11-77de-4d3f-abcf-b0b0ce617a9e-kube-api-access-xzv9q" (OuterVolumeSpecName: "kube-api-access-xzv9q") pod "67f04a11-77de-4d3f-abcf-b0b0ce617a9e" (UID: "67f04a11-77de-4d3f-abcf-b0b0ce617a9e"). InnerVolumeSpecName "kube-api-access-xzv9q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:09:41.080263 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:41.080235 2556 generic.go:356] "Generic (PLEG): container finished" podID="67f04a11-77de-4d3f-abcf-b0b0ce617a9e" containerID="5288c05dc2b7e0ddd85fa92e8523fe8553a58aba9209807b7f055b931f31f67b" exitCode=0 Jul 02 22:09:41.080403 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:41.080280 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-d372c0-7d6f54c987-tqd6v" Jul 02 22:09:41.080403 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:41.080290 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-d372c0-7d6f54c987-tqd6v" event={"ID":"67f04a11-77de-4d3f-abcf-b0b0ce617a9e","Type":"ContainerDied","Data":"5288c05dc2b7e0ddd85fa92e8523fe8553a58aba9209807b7f055b931f31f67b"} Jul 02 22:09:41.080403 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:41.080337 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-d372c0-7d6f54c987-tqd6v" event={"ID":"67f04a11-77de-4d3f-abcf-b0b0ce617a9e","Type":"ContainerDied","Data":"231442abc6b8d13cc10d55afe1c000ad4edb61d0b358caefeff68cf48d2886d4"} Jul 02 22:09:41.080403 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:41.080360 2556 scope.go:117] "RemoveContainer" containerID="5288c05dc2b7e0ddd85fa92e8523fe8553a58aba9209807b7f055b931f31f67b" Jul 02 22:09:41.088709 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:41.088692 2556 scope.go:117] "RemoveContainer" containerID="5288c05dc2b7e0ddd85fa92e8523fe8553a58aba9209807b7f055b931f31f67b" Jul 02 22:09:41.088940 ip-10-0-140-146 kubenswrapper[2556]: E0702 22:09:41.088923 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5288c05dc2b7e0ddd85fa92e8523fe8553a58aba9209807b7f055b931f31f67b\": container with ID starting with 5288c05dc2b7e0ddd85fa92e8523fe8553a58aba9209807b7f055b931f31f67b not found: ID does not exist" containerID="5288c05dc2b7e0ddd85fa92e8523fe8553a58aba9209807b7f055b931f31f67b" Jul 02 22:09:41.088997 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:41.088948 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5288c05dc2b7e0ddd85fa92e8523fe8553a58aba9209807b7f055b931f31f67b"} err="failed to get container status \"5288c05dc2b7e0ddd85fa92e8523fe8553a58aba9209807b7f055b931f31f67b\": rpc error: code = NotFound desc = could not find container \"5288c05dc2b7e0ddd85fa92e8523fe8553a58aba9209807b7f055b931f31f67b\": container with ID starting with 5288c05dc2b7e0ddd85fa92e8523fe8553a58aba9209807b7f055b931f31f67b not found: ID does not exist" Jul 02 22:09:41.102372 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:41.102316 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-d372c0-7d6f54c987-tqd6v"] Jul 02 22:09:41.104148 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:41.104127 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-d372c0-7d6f54c987-tqd6v"] Jul 02 22:09:41.146421 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:41.146393 2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67f04a11-77de-4d3f-abcf-b0b0ce617a9e-openshift-service-ca\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:09:41.146421 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:41.146419 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67f04a11-77de-4d3f-abcf-b0b0ce617a9e-maas-api-tls\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:09:41.146563 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:41.146429 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzv9q\" (UniqueName: \"kubernetes.io/projected/67f04a11-77de-4d3f-abcf-b0b0ce617a9e-kube-api-access-xzv9q\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:09:41.815334 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:41.815303 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67f04a11-77de-4d3f-abcf-b0b0ce617a9e" path="/var/lib/kubelet/pods/67f04a11-77de-4d3f-abcf-b0b0ce617a9e/volumes" Jul 02 22:09:44.133333 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:44.133298 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:09:46.303985 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:46.303957 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:46.543911 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:46.543858 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr"] Jul 02 22:09:46.544127 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:46.544103 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" podUID="c73068b3-2dd8-4c8c-bc74-5860f03aea13" containerName="istio-proxy" containerID="cri-o://c63b447889376da359a95e945201a922d915a31ebeafd3c23e9ef0854d0d34f3" gracePeriod=30 Jul 02 22:09:47.408345 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.408307 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl"] Jul 02 22:09:47.408788 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.408607 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67f04a11-77de-4d3f-abcf-b0b0ce617a9e" containerName="maas-api" Jul 02 22:09:47.408788 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.408619 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="67f04a11-77de-4d3f-abcf-b0b0ce617a9e" containerName="maas-api" Jul 02 22:09:47.408788 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.408628 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e137ddd0-010e-450e-bfee-2ed20e645259" containerName="maas-api" Jul 02 22:09:47.408788 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.408633 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e137ddd0-010e-450e-bfee-2ed20e645259" containerName="maas-api" Jul 02 22:09:47.408788 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.408672 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="67f04a11-77de-4d3f-abcf-b0b0ce617a9e" containerName="maas-api" Jul 02 22:09:47.408788 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.408678 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e137ddd0-010e-450e-bfee-2ed20e645259" containerName="maas-api" Jul 02 22:09:47.411217 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.411196 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:09:47.412093 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.412055 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:47.413944 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.413924 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-bd7159-openshift-default-dockercfg-g8spr\"" Jul 02 22:09:47.423979 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.423956 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl"] Jul 02 22:09:47.492352 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.492322 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a34d0412-2a2f-46df-97f8-46785985abbc-istio-envoy\") pod \"e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl\" (UID: \"a34d0412-2a2f-46df-97f8-46785985abbc\") " pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:09:47.492487 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.492355 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a34d0412-2a2f-46df-97f8-46785985abbc-workload-certs\") pod \"e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl\" (UID: \"a34d0412-2a2f-46df-97f8-46785985abbc\") " pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:09:47.492487 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.492385 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a34d0412-2a2f-46df-97f8-46785985abbc-istiod-ca-cert\") pod \"e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl\" (UID: \"a34d0412-2a2f-46df-97f8-46785985abbc\") " pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:09:47.492487 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.492429 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a34d0412-2a2f-46df-97f8-46785985abbc-istio-podinfo\") pod \"e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl\" (UID: \"a34d0412-2a2f-46df-97f8-46785985abbc\") " pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:09:47.492487 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.492461 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a34d0412-2a2f-46df-97f8-46785985abbc-workload-socket\") pod \"e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl\" (UID: \"a34d0412-2a2f-46df-97f8-46785985abbc\") " pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:09:47.492487 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.492477 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a34d0412-2a2f-46df-97f8-46785985abbc-istio-data\") pod \"e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl\" (UID: \"a34d0412-2a2f-46df-97f8-46785985abbc\") " pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:09:47.492651 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.492495 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfwp5\" (UniqueName: \"kubernetes.io/projected/a34d0412-2a2f-46df-97f8-46785985abbc-kube-api-access-gfwp5\") pod \"e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl\" (UID: \"a34d0412-2a2f-46df-97f8-46785985abbc\") " pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:09:47.492651 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.492546 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a34d0412-2a2f-46df-97f8-46785985abbc-istio-token\") pod \"e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl\" (UID: \"a34d0412-2a2f-46df-97f8-46785985abbc\") " pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:09:47.492651 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.492579 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a34d0412-2a2f-46df-97f8-46785985abbc-credential-socket\") pod \"e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl\" (UID: \"a34d0412-2a2f-46df-97f8-46785985abbc\") " pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:09:47.511099 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.511058 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:47.593328 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.593300 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a34d0412-2a2f-46df-97f8-46785985abbc-istio-envoy\") pod \"e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl\" (UID: \"a34d0412-2a2f-46df-97f8-46785985abbc\") " pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:09:47.593521 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.593338 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a34d0412-2a2f-46df-97f8-46785985abbc-workload-certs\") pod \"e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl\" (UID: \"a34d0412-2a2f-46df-97f8-46785985abbc\") " pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:09:47.593521 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.593369 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a34d0412-2a2f-46df-97f8-46785985abbc-istiod-ca-cert\") pod \"e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl\" (UID: \"a34d0412-2a2f-46df-97f8-46785985abbc\") " pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:09:47.593521 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.593396 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a34d0412-2a2f-46df-97f8-46785985abbc-istio-podinfo\") pod \"e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl\" (UID: \"a34d0412-2a2f-46df-97f8-46785985abbc\") " pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:09:47.593521 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.593423 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a34d0412-2a2f-46df-97f8-46785985abbc-workload-socket\") pod \"e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl\" (UID: \"a34d0412-2a2f-46df-97f8-46785985abbc\") " pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:09:47.593521 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.593447 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a34d0412-2a2f-46df-97f8-46785985abbc-istio-data\") pod \"e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl\" (UID: \"a34d0412-2a2f-46df-97f8-46785985abbc\") " pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:09:47.593521 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.593477 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfwp5\" (UniqueName: \"kubernetes.io/projected/a34d0412-2a2f-46df-97f8-46785985abbc-kube-api-access-gfwp5\") pod \"e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl\" (UID: \"a34d0412-2a2f-46df-97f8-46785985abbc\") " pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:09:47.593521 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.593519 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a34d0412-2a2f-46df-97f8-46785985abbc-istio-token\") pod \"e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl\" (UID: \"a34d0412-2a2f-46df-97f8-46785985abbc\") " pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:09:47.593906 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.593551 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a34d0412-2a2f-46df-97f8-46785985abbc-credential-socket\") pod \"e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl\" (UID: \"a34d0412-2a2f-46df-97f8-46785985abbc\") " pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:09:47.593969 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.593902 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a34d0412-2a2f-46df-97f8-46785985abbc-workload-certs\") pod \"e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl\" (UID: \"a34d0412-2a2f-46df-97f8-46785985abbc\") " pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:09:47.594023 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.593963 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a34d0412-2a2f-46df-97f8-46785985abbc-istio-data\") pod \"e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl\" (UID: \"a34d0412-2a2f-46df-97f8-46785985abbc\") " pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:09:47.594077 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.594022 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a34d0412-2a2f-46df-97f8-46785985abbc-credential-socket\") pod \"e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl\" (UID: \"a34d0412-2a2f-46df-97f8-46785985abbc\") " pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:09:47.594304 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.594276 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a34d0412-2a2f-46df-97f8-46785985abbc-workload-socket\") pod \"e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl\" (UID: \"a34d0412-2a2f-46df-97f8-46785985abbc\") " pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:09:47.594408 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.594392 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a34d0412-2a2f-46df-97f8-46785985abbc-istiod-ca-cert\") pod \"e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl\" (UID: \"a34d0412-2a2f-46df-97f8-46785985abbc\") " pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:09:47.595775 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.595755 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a34d0412-2a2f-46df-97f8-46785985abbc-istio-envoy\") pod \"e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl\" (UID: \"a34d0412-2a2f-46df-97f8-46785985abbc\") " pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:09:47.596205 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.596188 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a34d0412-2a2f-46df-97f8-46785985abbc-istio-podinfo\") pod \"e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl\" (UID: \"a34d0412-2a2f-46df-97f8-46785985abbc\") " pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:09:47.601536 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.601506 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a34d0412-2a2f-46df-97f8-46785985abbc-istio-token\") pod \"e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl\" (UID: \"a34d0412-2a2f-46df-97f8-46785985abbc\") " pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:09:47.601866 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.601846 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfwp5\" (UniqueName: \"kubernetes.io/projected/a34d0412-2a2f-46df-97f8-46785985abbc-kube-api-access-gfwp5\") pod \"e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl\" (UID: \"a34d0412-2a2f-46df-97f8-46785985abbc\") " pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:09:47.665906 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.665828 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:47.721173 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.721148 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:09:47.849101 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.849079 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl"] Jul 02 22:09:47.851587 ip-10-0-140-146 kubenswrapper[2556]: W0702 22:09:47.851562 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda34d0412_2a2f_46df_97f8_46785985abbc.slice/crio-aff5906d49ffa5ab2fb1a0745d655d5b4240f400e7c97e3c75e14c13f28c371c WatchSource:0}: Error finding container aff5906d49ffa5ab2fb1a0745d655d5b4240f400e7c97e3c75e14c13f28c371c: Status 404 returned error can't find the container with id aff5906d49ffa5ab2fb1a0745d655d5b4240f400e7c97e3c75e14c13f28c371c Jul 02 22:09:47.853664 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.853636 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 22:09:47.853725 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.853696 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 22:09:47.853766 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:47.853727 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 22:09:48.112657 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:48.112621 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" event={"ID":"a34d0412-2a2f-46df-97f8-46785985abbc","Type":"ContainerStarted","Data":"97033d1d9e219e7bb60f7d111c8fcedcf4edc9f754c25f3c6cb9df9c38fa0ecc"} Jul 02 22:09:48.112814 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:48.112686 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" event={"ID":"a34d0412-2a2f-46df-97f8-46785985abbc","Type":"ContainerStarted","Data":"aff5906d49ffa5ab2fb1a0745d655d5b4240f400e7c97e3c75e14c13f28c371c"} Jul 02 22:09:48.133193 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:48.133140 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" podStartSLOduration=1.133121637 podStartE2EDuration="1.133121637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:09:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:09:48.130960692 +0000 UTC m=+2024.850310223" watchObservedRunningTime="2026-07-02 22:09:48.133121637 +0000 UTC m=+2024.852471179" Jul 02 22:09:48.722032 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:48.721994 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:09:48.726764 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:48.726742 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:09:49.116559 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:49.116521 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:09:49.117455 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:49.117435 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:09:50.198465 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:50.198427 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:50.248730 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:50.248703 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-bd7159-855449d988-lxpmr"] Jul 02 22:09:50.251569 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:50.251548 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-bd7159-855449d988-lxpmr" Jul 02 22:09:50.254256 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:50.254235 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-bd7159-serving-cert\"" Jul 02 22:09:50.254256 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:50.254250 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-66gqh\"" Jul 02 22:09:50.262776 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:50.262754 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-bd7159-855449d988-lxpmr"] Jul 02 22:09:50.277217 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:50.277195 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:50.316307 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:50.316281 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10834012-b7ac-43d8-879e-110239862715-maas-api-tls\") pod \"maas-api-e2e-api-a-bd7159-855449d988-lxpmr\" (UID: \"10834012-b7ac-43d8-879e-110239862715\") " pod="opendatahub/maas-api-e2e-api-a-bd7159-855449d988-lxpmr" Jul 02 22:09:50.316392 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:50.316306 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10834012-b7ac-43d8-879e-110239862715-openshift-service-ca\") pod \"maas-api-e2e-api-a-bd7159-855449d988-lxpmr\" (UID: \"10834012-b7ac-43d8-879e-110239862715\") " pod="opendatahub/maas-api-e2e-api-a-bd7159-855449d988-lxpmr" Jul 02 22:09:50.316392 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:50.316324 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b688m\" (UniqueName: \"kubernetes.io/projected/10834012-b7ac-43d8-879e-110239862715-kube-api-access-b688m\") pod \"maas-api-e2e-api-a-bd7159-855449d988-lxpmr\" (UID: \"10834012-b7ac-43d8-879e-110239862715\") " pod="opendatahub/maas-api-e2e-api-a-bd7159-855449d988-lxpmr" Jul 02 22:09:50.337268 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:50.337246 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:50.417091 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:50.417060 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10834012-b7ac-43d8-879e-110239862715-maas-api-tls\") pod \"maas-api-e2e-api-a-bd7159-855449d988-lxpmr\" (UID: \"10834012-b7ac-43d8-879e-110239862715\") " pod="opendatahub/maas-api-e2e-api-a-bd7159-855449d988-lxpmr" Jul 02 22:09:50.417091 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:50.417090 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10834012-b7ac-43d8-879e-110239862715-openshift-service-ca\") pod \"maas-api-e2e-api-a-bd7159-855449d988-lxpmr\" (UID: \"10834012-b7ac-43d8-879e-110239862715\") " pod="opendatahub/maas-api-e2e-api-a-bd7159-855449d988-lxpmr" Jul 02 22:09:50.417293 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:50.417108 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b688m\" (UniqueName: \"kubernetes.io/projected/10834012-b7ac-43d8-879e-110239862715-kube-api-access-b688m\") pod \"maas-api-e2e-api-a-bd7159-855449d988-lxpmr\" (UID: \"10834012-b7ac-43d8-879e-110239862715\") " pod="opendatahub/maas-api-e2e-api-a-bd7159-855449d988-lxpmr" Jul 02 22:09:50.417796 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:50.417767 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10834012-b7ac-43d8-879e-110239862715-openshift-service-ca\") pod \"maas-api-e2e-api-a-bd7159-855449d988-lxpmr\" (UID: \"10834012-b7ac-43d8-879e-110239862715\") " pod="opendatahub/maas-api-e2e-api-a-bd7159-855449d988-lxpmr" Jul 02 22:09:50.419854 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:50.419832 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10834012-b7ac-43d8-879e-110239862715-maas-api-tls\") pod \"maas-api-e2e-api-a-bd7159-855449d988-lxpmr\" (UID: \"10834012-b7ac-43d8-879e-110239862715\") " pod="opendatahub/maas-api-e2e-api-a-bd7159-855449d988-lxpmr" Jul 02 22:09:50.426532 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:50.426508 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b688m\" (UniqueName: \"kubernetes.io/projected/10834012-b7ac-43d8-879e-110239862715-kube-api-access-b688m\") pod \"maas-api-e2e-api-a-bd7159-855449d988-lxpmr\" (UID: \"10834012-b7ac-43d8-879e-110239862715\") " pod="opendatahub/maas-api-e2e-api-a-bd7159-855449d988-lxpmr" Jul 02 22:09:50.456180 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:50.456118 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:50.561517 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:50.561488 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-bd7159-855449d988-lxpmr" Jul 02 22:09:50.683826 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:50.683800 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-bd7159-855449d988-lxpmr"] Jul 02 22:09:50.685755 ip-10-0-140-146 kubenswrapper[2556]: W0702 22:09:50.685728 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10834012_b7ac_43d8_879e_110239862715.slice/crio-4028046af432afe3fddd0fd98cb58a0b776bfa4579422d05d38c962b0653b726 WatchSource:0}: Error finding container 4028046af432afe3fddd0fd98cb58a0b776bfa4579422d05d38c962b0653b726: Status 404 returned error can't find the container with id 4028046af432afe3fddd0fd98cb58a0b776bfa4579422d05d38c962b0653b726 Jul 02 22:09:50.818587 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:50.818557 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:50.920685 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:50.920653 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:51.126176 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:51.126155 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-bd7159-855449d988-lxpmr" event={"ID":"10834012-b7ac-43d8-879e-110239862715","Type":"ContainerStarted","Data":"4028046af432afe3fddd0fd98cb58a0b776bfa4579422d05d38c962b0653b726"} Jul 02 22:09:51.783716 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:51.783698 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:09:51.830324 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:51.830260 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c73068b3-2dd8-4c8c-bc74-5860f03aea13-istio-podinfo\") pod \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\" (UID: \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\") " Jul 02 22:09:51.830324 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:51.830298 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c73068b3-2dd8-4c8c-bc74-5860f03aea13-credential-socket\") pod \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\" (UID: \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\") " Jul 02 22:09:51.830512 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:51.830325 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c73068b3-2dd8-4c8c-bc74-5860f03aea13-istio-token\") pod \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\" (UID: \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\") " Jul 02 22:09:51.830512 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:51.830372 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c73068b3-2dd8-4c8c-bc74-5860f03aea13-istio-envoy\") pod \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\" (UID: \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\") " Jul 02 22:09:51.830512 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:51.830399 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c73068b3-2dd8-4c8c-bc74-5860f03aea13-istio-data\") pod \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\" (UID: \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\") " Jul 02 22:09:51.830512 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:51.830428 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c73068b3-2dd8-4c8c-bc74-5860f03aea13-workload-socket\") pod \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\" (UID: \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\") " Jul 02 22:09:51.830780 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:51.830761 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c73068b3-2dd8-4c8c-bc74-5860f03aea13-workload-certs\") pod \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\" (UID: \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\") " Jul 02 22:09:51.830837 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:51.830515 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c73068b3-2dd8-4c8c-bc74-5860f03aea13-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c73068b3-2dd8-4c8c-bc74-5860f03aea13" (UID: "c73068b3-2dd8-4c8c-bc74-5860f03aea13"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:09:51.830916 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:51.830695 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c73068b3-2dd8-4c8c-bc74-5860f03aea13-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c73068b3-2dd8-4c8c-bc74-5860f03aea13" (UID: "c73068b3-2dd8-4c8c-bc74-5860f03aea13"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:09:51.830916 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:51.830810 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c73068b3-2dd8-4c8c-bc74-5860f03aea13-istiod-ca-cert\") pod \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\" (UID: \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\") " Jul 02 22:09:51.830916 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:51.830902 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fgm92\" (UniqueName: \"kubernetes.io/projected/c73068b3-2dd8-4c8c-bc74-5860f03aea13-kube-api-access-fgm92\") pod \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\" (UID: \"c73068b3-2dd8-4c8c-bc74-5860f03aea13\") " Jul 02 22:09:51.831169 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:51.831123 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c73068b3-2dd8-4c8c-bc74-5860f03aea13-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c73068b3-2dd8-4c8c-bc74-5860f03aea13" (UID: "c73068b3-2dd8-4c8c-bc74-5860f03aea13"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:09:51.831268 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:51.831039 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c73068b3-2dd8-4c8c-bc74-5860f03aea13-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c73068b3-2dd8-4c8c-bc74-5860f03aea13" (UID: "c73068b3-2dd8-4c8c-bc74-5860f03aea13"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:09:51.831359 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:51.831334 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c73068b3-2dd8-4c8c-bc74-5860f03aea13-workload-socket\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:09:51.831480 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:51.831361 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c73068b3-2dd8-4c8c-bc74-5860f03aea13-workload-certs\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:09:51.831480 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:51.831376 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c73068b3-2dd8-4c8c-bc74-5860f03aea13-istiod-ca-cert\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:09:51.831480 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:51.831390 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c73068b3-2dd8-4c8c-bc74-5860f03aea13-credential-socket\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:09:51.831641 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:51.831518 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c73068b3-2dd8-4c8c-bc74-5860f03aea13-istio-data" (OuterVolumeSpecName: "istio-data") pod "c73068b3-2dd8-4c8c-bc74-5860f03aea13" (UID: "c73068b3-2dd8-4c8c-bc74-5860f03aea13"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:09:51.832842 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:51.832821 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c73068b3-2dd8-4c8c-bc74-5860f03aea13-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c73068b3-2dd8-4c8c-bc74-5860f03aea13" (UID: "c73068b3-2dd8-4c8c-bc74-5860f03aea13"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:09:51.832842 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:51.832836 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c73068b3-2dd8-4c8c-bc74-5860f03aea13-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c73068b3-2dd8-4c8c-bc74-5860f03aea13" (UID: "c73068b3-2dd8-4c8c-bc74-5860f03aea13"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 22:09:51.832986 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:51.832836 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c73068b3-2dd8-4c8c-bc74-5860f03aea13-istio-token" (OuterVolumeSpecName: "istio-token") pod "c73068b3-2dd8-4c8c-bc74-5860f03aea13" (UID: "c73068b3-2dd8-4c8c-bc74-5860f03aea13"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:09:51.833276 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:51.833258 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c73068b3-2dd8-4c8c-bc74-5860f03aea13-kube-api-access-fgm92" (OuterVolumeSpecName: "kube-api-access-fgm92") pod "c73068b3-2dd8-4c8c-bc74-5860f03aea13" (UID: "c73068b3-2dd8-4c8c-bc74-5860f03aea13"). InnerVolumeSpecName "kube-api-access-fgm92". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:09:51.932516 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:51.932483 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c73068b3-2dd8-4c8c-bc74-5860f03aea13-istio-envoy\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:09:51.932516 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:51.932513 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c73068b3-2dd8-4c8c-bc74-5860f03aea13-istio-data\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:09:51.932516 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:51.932523 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fgm92\" (UniqueName: \"kubernetes.io/projected/c73068b3-2dd8-4c8c-bc74-5860f03aea13-kube-api-access-fgm92\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:09:51.932757 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:51.932532 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c73068b3-2dd8-4c8c-bc74-5860f03aea13-istio-podinfo\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:09:51.932757 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:51.932542 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c73068b3-2dd8-4c8c-bc74-5860f03aea13-istio-token\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:09:52.131302 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:52.131212 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-bd7159-855449d988-lxpmr" event={"ID":"10834012-b7ac-43d8-879e-110239862715","Type":"ContainerStarted","Data":"387b0c4a017e8d98933f13c404dcc4f1bbeaf6afd0de31af1e57bae9ebbab53d"} Jul 02 22:09:52.131302 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:52.131289 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-bd7159-855449d988-lxpmr" Jul 02 22:09:52.132817 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:52.132794 2556 generic.go:356] "Generic (PLEG): container finished" podID="c73068b3-2dd8-4c8c-bc74-5860f03aea13" containerID="c63b447889376da359a95e945201a922d915a31ebeafd3c23e9ef0854d0d34f3" exitCode=0 Jul 02 22:09:52.132956 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:52.132838 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" Jul 02 22:09:52.132956 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:52.132851 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" event={"ID":"c73068b3-2dd8-4c8c-bc74-5860f03aea13","Type":"ContainerDied","Data":"c63b447889376da359a95e945201a922d915a31ebeafd3c23e9ef0854d0d34f3"} Jul 02 22:09:52.132956 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:52.132914 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr" event={"ID":"c73068b3-2dd8-4c8c-bc74-5860f03aea13","Type":"ContainerDied","Data":"511d3718742811a02985c6fe247cbe3aeb299df58dd62dce8693fe122477b9be"} Jul 02 22:09:52.132956 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:52.132936 2556 scope.go:117] "RemoveContainer" containerID="c63b447889376da359a95e945201a922d915a31ebeafd3c23e9ef0854d0d34f3" Jul 02 22:09:52.141296 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:52.141281 2556 scope.go:117] "RemoveContainer" containerID="66d3a60c4e96bdf908335bde323a66300b3fc09c96f7870dc0561b5e0e244cce" Jul 02 22:09:52.148702 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:52.148684 2556 scope.go:117] "RemoveContainer" containerID="c63b447889376da359a95e945201a922d915a31ebeafd3c23e9ef0854d0d34f3" Jul 02 22:09:52.148959 ip-10-0-140-146 kubenswrapper[2556]: E0702 22:09:52.148939 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c63b447889376da359a95e945201a922d915a31ebeafd3c23e9ef0854d0d34f3\": container with ID starting with c63b447889376da359a95e945201a922d915a31ebeafd3c23e9ef0854d0d34f3 not found: ID does not exist" containerID="c63b447889376da359a95e945201a922d915a31ebeafd3c23e9ef0854d0d34f3" Jul 02 22:09:52.149053 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:52.148969 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c63b447889376da359a95e945201a922d915a31ebeafd3c23e9ef0854d0d34f3"} err="failed to get container status \"c63b447889376da359a95e945201a922d915a31ebeafd3c23e9ef0854d0d34f3\": rpc error: code = NotFound desc = could not find container \"c63b447889376da359a95e945201a922d915a31ebeafd3c23e9ef0854d0d34f3\": container with ID starting with c63b447889376da359a95e945201a922d915a31ebeafd3c23e9ef0854d0d34f3 not found: ID does not exist" Jul 02 22:09:52.149053 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:52.148991 2556 scope.go:117] "RemoveContainer" containerID="66d3a60c4e96bdf908335bde323a66300b3fc09c96f7870dc0561b5e0e244cce" Jul 02 22:09:52.149240 ip-10-0-140-146 kubenswrapper[2556]: E0702 22:09:52.149213 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66d3a60c4e96bdf908335bde323a66300b3fc09c96f7870dc0561b5e0e244cce\": container with ID starting with 66d3a60c4e96bdf908335bde323a66300b3fc09c96f7870dc0561b5e0e244cce not found: ID does not exist" containerID="66d3a60c4e96bdf908335bde323a66300b3fc09c96f7870dc0561b5e0e244cce" Jul 02 22:09:52.149378 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:52.149249 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66d3a60c4e96bdf908335bde323a66300b3fc09c96f7870dc0561b5e0e244cce"} err="failed to get container status \"66d3a60c4e96bdf908335bde323a66300b3fc09c96f7870dc0561b5e0e244cce\": rpc error: code = NotFound desc = could not find container \"66d3a60c4e96bdf908335bde323a66300b3fc09c96f7870dc0561b5e0e244cce\": container with ID starting with 66d3a60c4e96bdf908335bde323a66300b3fc09c96f7870dc0561b5e0e244cce not found: ID does not exist" Jul 02 22:09:52.150324 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:52.150289 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-bd7159-855449d988-lxpmr" podStartSLOduration=1.725522712 podStartE2EDuration="2.150276354s" totalImagesPullingTime="424.753642ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:09:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:09:52.148766947 +0000 UTC m=+2028.868116489" watchObservedRunningTime="2026-07-02 22:09:52.150276354 +0000 UTC m=+2028.869625894" Jul 02 22:09:52.163112 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:52.163091 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr"] Jul 02 22:09:52.166585 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:52.166565 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-d372c0-openshift-default-84498479b5-x8ljr"] Jul 02 22:09:53.815490 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:53.815457 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c73068b3-2dd8-4c8c-bc74-5860f03aea13" path="/var/lib/kubelet/pods/c73068b3-2dd8-4c8c-bc74-5860f03aea13/volumes" Jul 02 22:09:54.683592 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:54.683555 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:55.153555 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:55.153522 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:55.434789 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:55.434715 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:55.559100 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:55.559065 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:57.634642 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:57.634610 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:09:58.143250 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:58.143222 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-bd7159-855449d988-lxpmr" Jul 02 22:09:58.598422 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:58.598386 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:58.690292 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:58.690260 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:58.794901 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:58.792766 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt"] Jul 02 22:09:58.794901 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:58.793277 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c73068b3-2dd8-4c8c-bc74-5860f03aea13" containerName="istio-proxy" Jul 02 22:09:58.794901 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:58.793293 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="c73068b3-2dd8-4c8c-bc74-5860f03aea13" containerName="istio-proxy" Jul 02 22:09:58.794901 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:58.793380 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="c73068b3-2dd8-4c8c-bc74-5860f03aea13" containerName="istio-proxy" Jul 02 22:09:58.798203 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:58.797334 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt" Jul 02 22:09:58.805338 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:58.805307 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt"] Jul 02 22:09:58.805771 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:58.805751 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-906c71-serving-cert\"" Jul 02 22:09:58.829457 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:58.829417 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:58.907946 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:58.907692 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb31cfc4-33c5-412f-a457-ad10c9a587f8-maas-api-tls\") pod \"maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt\" (UID: \"fb31cfc4-33c5-412f-a457-ad10c9a587f8\") " pod="opendatahub/maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt" Jul 02 22:09:58.907946 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:58.907749 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhpmm\" (UniqueName: \"kubernetes.io/projected/fb31cfc4-33c5-412f-a457-ad10c9a587f8-kube-api-access-bhpmm\") pod \"maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt\" (UID: \"fb31cfc4-33c5-412f-a457-ad10c9a587f8\") " pod="opendatahub/maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt" Jul 02 22:09:58.907946 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:58.907807 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb31cfc4-33c5-412f-a457-ad10c9a587f8-openshift-service-ca\") pod \"maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt\" (UID: \"fb31cfc4-33c5-412f-a457-ad10c9a587f8\") " pod="opendatahub/maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt" Jul 02 22:09:59.008234 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:59.008194 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb31cfc4-33c5-412f-a457-ad10c9a587f8-openshift-service-ca\") pod \"maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt\" (UID: \"fb31cfc4-33c5-412f-a457-ad10c9a587f8\") " pod="opendatahub/maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt" Jul 02 22:09:59.008416 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:59.008262 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb31cfc4-33c5-412f-a457-ad10c9a587f8-maas-api-tls\") pod \"maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt\" (UID: \"fb31cfc4-33c5-412f-a457-ad10c9a587f8\") " pod="opendatahub/maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt" Jul 02 22:09:59.008416 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:59.008283 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhpmm\" (UniqueName: \"kubernetes.io/projected/fb31cfc4-33c5-412f-a457-ad10c9a587f8-kube-api-access-bhpmm\") pod \"maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt\" (UID: \"fb31cfc4-33c5-412f-a457-ad10c9a587f8\") " pod="opendatahub/maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt" Jul 02 22:09:59.008567 ip-10-0-140-146 kubenswrapper[2556]: E0702 22:09:59.008436 2556 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-906c71-serving-cert: secret "maas-api-e2e-api-b-906c71-serving-cert" not found Jul 02 22:09:59.008567 ip-10-0-140-146 kubenswrapper[2556]: E0702 22:09:59.008527 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb31cfc4-33c5-412f-a457-ad10c9a587f8-maas-api-tls podName:fb31cfc4-33c5-412f-a457-ad10c9a587f8 nodeName:}" failed. No retries permitted until 2026-07-02 22:09:59.508506053 +0000 UTC m=+2036.227855586 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fb31cfc4-33c5-412f-a457-ad10c9a587f8-maas-api-tls") pod "maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt" (UID: "fb31cfc4-33c5-412f-a457-ad10c9a587f8") : secret "maas-api-e2e-api-b-906c71-serving-cert" not found Jul 02 22:09:59.008942 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:59.008921 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb31cfc4-33c5-412f-a457-ad10c9a587f8-openshift-service-ca\") pod \"maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt\" (UID: \"fb31cfc4-33c5-412f-a457-ad10c9a587f8\") " pod="opendatahub/maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt" Jul 02 22:09:59.017502 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:59.017483 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhpmm\" (UniqueName: \"kubernetes.io/projected/fb31cfc4-33c5-412f-a457-ad10c9a587f8-kube-api-access-bhpmm\") pod \"maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt\" (UID: \"fb31cfc4-33c5-412f-a457-ad10c9a587f8\") " pod="opendatahub/maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt" Jul 02 22:09:59.071635 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:59.071596 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:59.344999 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:59.344971 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:59.447354 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:59.447317 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:09:59.513023 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:59.512994 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb31cfc4-33c5-412f-a457-ad10c9a587f8-maas-api-tls\") pod \"maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt\" (UID: \"fb31cfc4-33c5-412f-a457-ad10c9a587f8\") " pod="opendatahub/maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt" Jul 02 22:09:59.515530 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:59.515499 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb31cfc4-33c5-412f-a457-ad10c9a587f8-maas-api-tls\") pod \"maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt\" (UID: \"fb31cfc4-33c5-412f-a457-ad10c9a587f8\") " pod="opendatahub/maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt" Jul 02 22:09:59.724337 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:59.724240 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt" Jul 02 22:09:59.874950 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:09:59.874917 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt"] Jul 02 22:09:59.878022 ip-10-0-140-146 kubenswrapper[2556]: W0702 22:09:59.877995 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb31cfc4_33c5_412f_a457_ad10c9a587f8.slice/crio-cff8d6c5865ca13ec95a27a8d32dd40e483ef2bd51fb61b20534f113d9caf599 WatchSource:0}: Error finding container cff8d6c5865ca13ec95a27a8d32dd40e483ef2bd51fb61b20534f113d9caf599: Status 404 returned error can't find the container with id cff8d6c5865ca13ec95a27a8d32dd40e483ef2bd51fb61b20534f113d9caf599 Jul 02 22:10:00.166391 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:00.166353 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt" event={"ID":"fb31cfc4-33c5-412f-a457-ad10c9a587f8","Type":"ContainerStarted","Data":"cff8d6c5865ca13ec95a27a8d32dd40e483ef2bd51fb61b20534f113d9caf599"} Jul 02 22:10:01.174906 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:01.172749 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt" event={"ID":"fb31cfc4-33c5-412f-a457-ad10c9a587f8","Type":"ContainerStarted","Data":"80875c16c5d7aae819d9e0dbefcebdbb588c845ad9c30621a3fc825d5cfefc49"} Jul 02 22:10:01.174906 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:01.172897 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt" Jul 02 22:10:01.212986 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:01.212932 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt" podStartSLOduration=2.771539978 podStartE2EDuration="3.212915989s" totalImagesPullingTime="441.376011ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:09:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:10:01.211550122 +0000 UTC m=+2037.930899663" watchObservedRunningTime="2026-07-02 22:10:01.212915989 +0000 UTC m=+2037.932265541" Jul 02 22:10:03.086272 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:03.086234 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:05.158021 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:05.157982 2556 patch_prober.go:28] interesting pod/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 02 22:10:05.158459 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:05.158042 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" podUID="a34d0412-2a2f-46df-97f8-46785985abbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 02 22:10:06.194690 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:06.194659 2556 generic.go:356] "Generic (PLEG): container finished" podID="a34d0412-2a2f-46df-97f8-46785985abbc" containerID="97033d1d9e219e7bb60f7d111c8fcedcf4edc9f754c25f3c6cb9df9c38fa0ecc" exitCode=0 Jul 02 22:10:06.195220 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:06.194724 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" event={"ID":"a34d0412-2a2f-46df-97f8-46785985abbc","Type":"ContainerDied","Data":"97033d1d9e219e7bb60f7d111c8fcedcf4edc9f754c25f3c6cb9df9c38fa0ecc"} Jul 02 22:10:06.195220 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:06.195189 2556 scope.go:117] "RemoveContainer" containerID="97033d1d9e219e7bb60f7d111c8fcedcf4edc9f754c25f3c6cb9df9c38fa0ecc" Jul 02 22:10:06.196185 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:06.196154 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 22:10:06.196289 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:06.196223 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 22:10:06.196289 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:06.196257 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 22:10:07.182413 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:07.182382 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt" Jul 02 22:10:07.201121 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:07.201088 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" event={"ID":"a34d0412-2a2f-46df-97f8-46785985abbc","Type":"ContainerStarted","Data":"583a3f000d9548e1a899cd2fe05f1a50ff08199784dd69bb3ec51c6cd1297071"} Jul 02 22:10:07.721885 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:07.721832 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:10:07.722107 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:07.722086 2556 patch_prober.go:28] interesting pod/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 02 22:10:07.722193 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:07.722138 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" podUID="a34d0412-2a2f-46df-97f8-46785985abbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 02 22:10:08.721704 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:08.721670 2556 patch_prober.go:28] interesting pod/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 02 22:10:08.722119 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:08.721734 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" podUID="a34d0412-2a2f-46df-97f8-46785985abbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 02 22:10:09.722342 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:09.722311 2556 patch_prober.go:28] interesting pod/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 02 22:10:09.722724 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:09.722374 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" podUID="a34d0412-2a2f-46df-97f8-46785985abbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 02 22:10:10.666512 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:10.666480 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:10.721884 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:10.721839 2556 patch_prober.go:28] interesting pod/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 02 22:10:10.722049 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:10.721916 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" podUID="a34d0412-2a2f-46df-97f8-46785985abbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 02 22:10:10.770000 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:10.769969 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:11.198515 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:11.198474 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:11.533806 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:11.533755 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:11.673330 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:11.673295 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:11.726086 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:11.726049 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:10:11.726401 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:11.726385 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:10:11.727142 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:11.727120 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:10:11.756160 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:11.756123 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:11.924382 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:11.924300 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-bd7159-855449d988-lxpmr"] Jul 02 22:10:11.924787 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:11.924534 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-bd7159-855449d988-lxpmr" podUID="10834012-b7ac-43d8-879e-110239862715" containerName="maas-api" containerID="cri-o://387b0c4a017e8d98933f13c404dcc4f1bbeaf6afd0de31af1e57bae9ebbab53d" gracePeriod=30 Jul 02 22:10:11.946938 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:11.946909 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:12.161101 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:12.161080 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-bd7159-855449d988-lxpmr" Jul 02 22:10:12.223040 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:12.222951 2556 generic.go:356] "Generic (PLEG): container finished" podID="10834012-b7ac-43d8-879e-110239862715" containerID="387b0c4a017e8d98933f13c404dcc4f1bbeaf6afd0de31af1e57bae9ebbab53d" exitCode=0 Jul 02 22:10:12.223040 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:12.222998 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-bd7159-855449d988-lxpmr" Jul 02 22:10:12.223265 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:12.223048 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-bd7159-855449d988-lxpmr" event={"ID":"10834012-b7ac-43d8-879e-110239862715","Type":"ContainerDied","Data":"387b0c4a017e8d98933f13c404dcc4f1bbeaf6afd0de31af1e57bae9ebbab53d"} Jul 02 22:10:12.223265 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:12.223105 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-bd7159-855449d988-lxpmr" event={"ID":"10834012-b7ac-43d8-879e-110239862715","Type":"ContainerDied","Data":"4028046af432afe3fddd0fd98cb58a0b776bfa4579422d05d38c962b0653b726"} Jul 02 22:10:12.223265 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:12.223138 2556 scope.go:117] "RemoveContainer" containerID="387b0c4a017e8d98933f13c404dcc4f1bbeaf6afd0de31af1e57bae9ebbab53d" Jul 02 22:10:12.231493 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:12.231479 2556 scope.go:117] "RemoveContainer" containerID="387b0c4a017e8d98933f13c404dcc4f1bbeaf6afd0de31af1e57bae9ebbab53d" Jul 02 22:10:12.231737 ip-10-0-140-146 kubenswrapper[2556]: E0702 22:10:12.231719 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"387b0c4a017e8d98933f13c404dcc4f1bbeaf6afd0de31af1e57bae9ebbab53d\": container with ID starting with 387b0c4a017e8d98933f13c404dcc4f1bbeaf6afd0de31af1e57bae9ebbab53d not found: ID does not exist" containerID="387b0c4a017e8d98933f13c404dcc4f1bbeaf6afd0de31af1e57bae9ebbab53d" Jul 02 22:10:12.231789 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:12.231747 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"387b0c4a017e8d98933f13c404dcc4f1bbeaf6afd0de31af1e57bae9ebbab53d"} err="failed to get container status \"387b0c4a017e8d98933f13c404dcc4f1bbeaf6afd0de31af1e57bae9ebbab53d\": rpc error: code = NotFound desc = could not find container \"387b0c4a017e8d98933f13c404dcc4f1bbeaf6afd0de31af1e57bae9ebbab53d\": container with ID starting with 387b0c4a017e8d98933f13c404dcc4f1bbeaf6afd0de31af1e57bae9ebbab53d not found: ID does not exist" Jul 02 22:10:12.327314 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:12.327291 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10834012-b7ac-43d8-879e-110239862715-maas-api-tls\") pod \"10834012-b7ac-43d8-879e-110239862715\" (UID: \"10834012-b7ac-43d8-879e-110239862715\") " Jul 02 22:10:12.327415 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:12.327343 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10834012-b7ac-43d8-879e-110239862715-openshift-service-ca\") pod \"10834012-b7ac-43d8-879e-110239862715\" (UID: \"10834012-b7ac-43d8-879e-110239862715\") " Jul 02 22:10:12.327415 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:12.327389 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b688m\" (UniqueName: \"kubernetes.io/projected/10834012-b7ac-43d8-879e-110239862715-kube-api-access-b688m\") pod \"10834012-b7ac-43d8-879e-110239862715\" (UID: \"10834012-b7ac-43d8-879e-110239862715\") " Jul 02 22:10:12.327752 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:12.327721 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10834012-b7ac-43d8-879e-110239862715-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "10834012-b7ac-43d8-879e-110239862715" (UID: "10834012-b7ac-43d8-879e-110239862715"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:10:12.329538 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:12.329517 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10834012-b7ac-43d8-879e-110239862715-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "10834012-b7ac-43d8-879e-110239862715" (UID: "10834012-b7ac-43d8-879e-110239862715"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 22:10:12.329635 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:12.329540 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10834012-b7ac-43d8-879e-110239862715-kube-api-access-b688m" (OuterVolumeSpecName: "kube-api-access-b688m") pod "10834012-b7ac-43d8-879e-110239862715" (UID: "10834012-b7ac-43d8-879e-110239862715"). InnerVolumeSpecName "kube-api-access-b688m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:10:12.428837 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:12.428789 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b688m\" (UniqueName: \"kubernetes.io/projected/10834012-b7ac-43d8-879e-110239862715-kube-api-access-b688m\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:10:12.428837 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:12.428831 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10834012-b7ac-43d8-879e-110239862715-maas-api-tls\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:10:12.428837 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:12.428841 2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10834012-b7ac-43d8-879e-110239862715-openshift-service-ca\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:10:12.545096 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:12.545067 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-bd7159-855449d988-lxpmr"] Jul 02 22:10:12.548547 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:12.548519 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-bd7159-855449d988-lxpmr"] Jul 02 22:10:13.815726 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:13.815695 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10834012-b7ac-43d8-879e-110239862715" path="/var/lib/kubelet/pods/10834012-b7ac-43d8-879e-110239862715/volumes" Jul 02 22:10:14.622664 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:14.622624 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:10:17.183043 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:17.183014 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:17.290025 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:17.289991 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:17.411751 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:17.411713 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl"] Jul 02 22:10:17.412020 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:17.411992 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" podUID="a34d0412-2a2f-46df-97f8-46785985abbc" containerName="istio-proxy" containerID="cri-o://583a3f000d9548e1a899cd2fe05f1a50ff08199784dd69bb3ec51c6cd1297071" gracePeriod=30 Jul 02 22:10:17.437839 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:17.437762 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:17.660352 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:17.660324 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:17.835429 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:17.835397 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:18.256875 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:18.256851 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:18.428640 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:18.428610 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:18.592681 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:18.592606 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:18.743513 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:18.743481 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt"] Jul 02 22:10:18.743761 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:18.743733 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt" podUID="fb31cfc4-33c5-412f-a457-ad10c9a587f8" containerName="maas-api" containerID="cri-o://80875c16c5d7aae819d9e0dbefcebdbb588c845ad9c30621a3fc825d5cfefc49" gracePeriod=30 Jul 02 22:10:18.781532 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:18.781504 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:18.993573 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:18.993551 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt" Jul 02 22:10:19.179926 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:19.179808 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb31cfc4-33c5-412f-a457-ad10c9a587f8-openshift-service-ca\") pod \"fb31cfc4-33c5-412f-a457-ad10c9a587f8\" (UID: \"fb31cfc4-33c5-412f-a457-ad10c9a587f8\") " Jul 02 22:10:19.179926 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:19.179855 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb31cfc4-33c5-412f-a457-ad10c9a587f8-maas-api-tls\") pod \"fb31cfc4-33c5-412f-a457-ad10c9a587f8\" (UID: \"fb31cfc4-33c5-412f-a457-ad10c9a587f8\") " Jul 02 22:10:19.180149 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:19.179958 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhpmm\" (UniqueName: \"kubernetes.io/projected/fb31cfc4-33c5-412f-a457-ad10c9a587f8-kube-api-access-bhpmm\") pod \"fb31cfc4-33c5-412f-a457-ad10c9a587f8\" (UID: \"fb31cfc4-33c5-412f-a457-ad10c9a587f8\") " Jul 02 22:10:19.180204 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:19.180171 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb31cfc4-33c5-412f-a457-ad10c9a587f8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fb31cfc4-33c5-412f-a457-ad10c9a587f8" (UID: "fb31cfc4-33c5-412f-a457-ad10c9a587f8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:10:19.182201 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:19.182171 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb31cfc4-33c5-412f-a457-ad10c9a587f8-kube-api-access-bhpmm" (OuterVolumeSpecName: "kube-api-access-bhpmm") pod "fb31cfc4-33c5-412f-a457-ad10c9a587f8" (UID: "fb31cfc4-33c5-412f-a457-ad10c9a587f8"). InnerVolumeSpecName "kube-api-access-bhpmm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:10:19.182201 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:19.182195 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb31cfc4-33c5-412f-a457-ad10c9a587f8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fb31cfc4-33c5-412f-a457-ad10c9a587f8" (UID: "fb31cfc4-33c5-412f-a457-ad10c9a587f8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 22:10:19.250938 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:19.250911 2556 generic.go:356] "Generic (PLEG): container finished" podID="fb31cfc4-33c5-412f-a457-ad10c9a587f8" containerID="80875c16c5d7aae819d9e0dbefcebdbb588c845ad9c30621a3fc825d5cfefc49" exitCode=0 Jul 02 22:10:19.251056 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:19.250950 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt" event={"ID":"fb31cfc4-33c5-412f-a457-ad10c9a587f8","Type":"ContainerDied","Data":"80875c16c5d7aae819d9e0dbefcebdbb588c845ad9c30621a3fc825d5cfefc49"} Jul 02 22:10:19.251056 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:19.250955 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt" Jul 02 22:10:19.251056 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:19.250988 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt" event={"ID":"fb31cfc4-33c5-412f-a457-ad10c9a587f8","Type":"ContainerDied","Data":"cff8d6c5865ca13ec95a27a8d32dd40e483ef2bd51fb61b20534f113d9caf599"} Jul 02 22:10:19.251056 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:19.251010 2556 scope.go:117] "RemoveContainer" containerID="80875c16c5d7aae819d9e0dbefcebdbb588c845ad9c30621a3fc825d5cfefc49" Jul 02 22:10:19.258833 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:19.258818 2556 scope.go:117] "RemoveContainer" containerID="80875c16c5d7aae819d9e0dbefcebdbb588c845ad9c30621a3fc825d5cfefc49" Jul 02 22:10:19.259101 ip-10-0-140-146 kubenswrapper[2556]: E0702 22:10:19.259053 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80875c16c5d7aae819d9e0dbefcebdbb588c845ad9c30621a3fc825d5cfefc49\": container with ID starting with 80875c16c5d7aae819d9e0dbefcebdbb588c845ad9c30621a3fc825d5cfefc49 not found: ID does not exist" containerID="80875c16c5d7aae819d9e0dbefcebdbb588c845ad9c30621a3fc825d5cfefc49" Jul 02 22:10:19.259101 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:19.259078 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80875c16c5d7aae819d9e0dbefcebdbb588c845ad9c30621a3fc825d5cfefc49"} err="failed to get container status \"80875c16c5d7aae819d9e0dbefcebdbb588c845ad9c30621a3fc825d5cfefc49\": rpc error: code = NotFound desc = could not find container \"80875c16c5d7aae819d9e0dbefcebdbb588c845ad9c30621a3fc825d5cfefc49\": container with ID starting with 80875c16c5d7aae819d9e0dbefcebdbb588c845ad9c30621a3fc825d5cfefc49 not found: ID does not exist" Jul 02 22:10:19.275082 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:19.275043 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt"] Jul 02 22:10:19.278792 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:19.278775 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-906c71-5cb97789bc-pfrbt"] Jul 02 22:10:19.280800 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:19.280780 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb31cfc4-33c5-412f-a457-ad10c9a587f8-maas-api-tls\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:10:19.280850 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:19.280805 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhpmm\" (UniqueName: \"kubernetes.io/projected/fb31cfc4-33c5-412f-a457-ad10c9a587f8-kube-api-access-bhpmm\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:10:19.280850 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:19.280815 2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb31cfc4-33c5-412f-a457-ad10c9a587f8-openshift-service-ca\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:10:19.815780 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:19.815737 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb31cfc4-33c5-412f-a457-ad10c9a587f8" path="/var/lib/kubelet/pods/fb31cfc4-33c5-412f-a457-ad10c9a587f8/volumes" Jul 02 22:10:22.656859 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:22.656837 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:10:22.806858 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:22.806830 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a34d0412-2a2f-46df-97f8-46785985abbc-istiod-ca-cert\") pod \"a34d0412-2a2f-46df-97f8-46785985abbc\" (UID: \"a34d0412-2a2f-46df-97f8-46785985abbc\") " Jul 02 22:10:22.807021 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:22.806862 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a34d0412-2a2f-46df-97f8-46785985abbc-istio-podinfo\") pod \"a34d0412-2a2f-46df-97f8-46785985abbc\" (UID: \"a34d0412-2a2f-46df-97f8-46785985abbc\") " Jul 02 22:10:22.807021 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:22.806915 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a34d0412-2a2f-46df-97f8-46785985abbc-istio-data\") pod \"a34d0412-2a2f-46df-97f8-46785985abbc\" (UID: \"a34d0412-2a2f-46df-97f8-46785985abbc\") " Jul 02 22:10:22.807021 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:22.806943 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a34d0412-2a2f-46df-97f8-46785985abbc-istio-token\") pod \"a34d0412-2a2f-46df-97f8-46785985abbc\" (UID: \"a34d0412-2a2f-46df-97f8-46785985abbc\") " Jul 02 22:10:22.807021 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:22.806985 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a34d0412-2a2f-46df-97f8-46785985abbc-istio-envoy\") pod \"a34d0412-2a2f-46df-97f8-46785985abbc\" (UID: \"a34d0412-2a2f-46df-97f8-46785985abbc\") " Jul 02 22:10:22.807241 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:22.807034 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a34d0412-2a2f-46df-97f8-46785985abbc-credential-socket\") pod \"a34d0412-2a2f-46df-97f8-46785985abbc\" (UID: \"a34d0412-2a2f-46df-97f8-46785985abbc\") " Jul 02 22:10:22.807241 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:22.807083 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a34d0412-2a2f-46df-97f8-46785985abbc-workload-socket\") pod \"a34d0412-2a2f-46df-97f8-46785985abbc\" (UID: \"a34d0412-2a2f-46df-97f8-46785985abbc\") " Jul 02 22:10:22.807241 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:22.807107 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a34d0412-2a2f-46df-97f8-46785985abbc-workload-certs\") pod \"a34d0412-2a2f-46df-97f8-46785985abbc\" (UID: \"a34d0412-2a2f-46df-97f8-46785985abbc\") " Jul 02 22:10:22.807241 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:22.807131 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfwp5\" (UniqueName: \"kubernetes.io/projected/a34d0412-2a2f-46df-97f8-46785985abbc-kube-api-access-gfwp5\") pod \"a34d0412-2a2f-46df-97f8-46785985abbc\" (UID: \"a34d0412-2a2f-46df-97f8-46785985abbc\") " Jul 02 22:10:22.807447 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:22.807258 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a34d0412-2a2f-46df-97f8-46785985abbc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a34d0412-2a2f-46df-97f8-46785985abbc" (UID: "a34d0412-2a2f-46df-97f8-46785985abbc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:10:22.807447 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:22.807394 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a34d0412-2a2f-46df-97f8-46785985abbc-istiod-ca-cert\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:10:22.807447 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:22.807409 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a34d0412-2a2f-46df-97f8-46785985abbc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a34d0412-2a2f-46df-97f8-46785985abbc" (UID: "a34d0412-2a2f-46df-97f8-46785985abbc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:10:22.807604 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:22.807582 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a34d0412-2a2f-46df-97f8-46785985abbc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a34d0412-2a2f-46df-97f8-46785985abbc" (UID: "a34d0412-2a2f-46df-97f8-46785985abbc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:10:22.807670 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:22.807638 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a34d0412-2a2f-46df-97f8-46785985abbc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a34d0412-2a2f-46df-97f8-46785985abbc" (UID: "a34d0412-2a2f-46df-97f8-46785985abbc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:10:22.808202 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:22.808176 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a34d0412-2a2f-46df-97f8-46785985abbc-istio-data" (OuterVolumeSpecName: "istio-data") pod "a34d0412-2a2f-46df-97f8-46785985abbc" (UID: "a34d0412-2a2f-46df-97f8-46785985abbc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:10:22.809363 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:22.809336 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a34d0412-2a2f-46df-97f8-46785985abbc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a34d0412-2a2f-46df-97f8-46785985abbc" (UID: "a34d0412-2a2f-46df-97f8-46785985abbc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 22:10:22.809589 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:22.809471 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a34d0412-2a2f-46df-97f8-46785985abbc-istio-token" (OuterVolumeSpecName: "istio-token") pod "a34d0412-2a2f-46df-97f8-46785985abbc" (UID: "a34d0412-2a2f-46df-97f8-46785985abbc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:10:22.809698 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:22.809678 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a34d0412-2a2f-46df-97f8-46785985abbc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a34d0412-2a2f-46df-97f8-46785985abbc" (UID: "a34d0412-2a2f-46df-97f8-46785985abbc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:10:22.809738 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:22.809692 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a34d0412-2a2f-46df-97f8-46785985abbc-kube-api-access-gfwp5" (OuterVolumeSpecName: "kube-api-access-gfwp5") pod "a34d0412-2a2f-46df-97f8-46785985abbc" (UID: "a34d0412-2a2f-46df-97f8-46785985abbc"). InnerVolumeSpecName "kube-api-access-gfwp5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:10:22.908002 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:22.907971 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a34d0412-2a2f-46df-97f8-46785985abbc-workload-certs\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:10:22.908002 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:22.907998 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfwp5\" (UniqueName: \"kubernetes.io/projected/a34d0412-2a2f-46df-97f8-46785985abbc-kube-api-access-gfwp5\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:10:22.908168 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:22.908011 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a34d0412-2a2f-46df-97f8-46785985abbc-istio-podinfo\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:10:22.908168 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:22.908021 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a34d0412-2a2f-46df-97f8-46785985abbc-istio-data\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:10:22.908168 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:22.908029 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a34d0412-2a2f-46df-97f8-46785985abbc-istio-token\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:10:22.908168 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:22.908036 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a34d0412-2a2f-46df-97f8-46785985abbc-istio-envoy\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:10:22.908168 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:22.908045 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a34d0412-2a2f-46df-97f8-46785985abbc-credential-socket\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:10:22.908168 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:22.908056 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a34d0412-2a2f-46df-97f8-46785985abbc-workload-socket\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:10:23.267887 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:23.267838 2556 generic.go:356] "Generic (PLEG): container finished" podID="a34d0412-2a2f-46df-97f8-46785985abbc" containerID="583a3f000d9548e1a899cd2fe05f1a50ff08199784dd69bb3ec51c6cd1297071" exitCode=0 Jul 02 22:10:23.268037 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:23.267900 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" event={"ID":"a34d0412-2a2f-46df-97f8-46785985abbc","Type":"ContainerDied","Data":"583a3f000d9548e1a899cd2fe05f1a50ff08199784dd69bb3ec51c6cd1297071"} Jul 02 22:10:23.268037 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:23.267905 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" Jul 02 22:10:23.268037 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:23.267944 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl" event={"ID":"a34d0412-2a2f-46df-97f8-46785985abbc","Type":"ContainerDied","Data":"aff5906d49ffa5ab2fb1a0745d655d5b4240f400e7c97e3c75e14c13f28c371c"} Jul 02 22:10:23.268037 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:23.267967 2556 scope.go:117] "RemoveContainer" containerID="583a3f000d9548e1a899cd2fe05f1a50ff08199784dd69bb3ec51c6cd1297071" Jul 02 22:10:23.276616 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:23.276597 2556 scope.go:117] "RemoveContainer" containerID="97033d1d9e219e7bb60f7d111c8fcedcf4edc9f754c25f3c6cb9df9c38fa0ecc" Jul 02 22:10:23.283903 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:23.283888 2556 scope.go:117] "RemoveContainer" containerID="583a3f000d9548e1a899cd2fe05f1a50ff08199784dd69bb3ec51c6cd1297071" Jul 02 22:10:23.284175 ip-10-0-140-146 kubenswrapper[2556]: E0702 22:10:23.284156 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"583a3f000d9548e1a899cd2fe05f1a50ff08199784dd69bb3ec51c6cd1297071\": container with ID starting with 583a3f000d9548e1a899cd2fe05f1a50ff08199784dd69bb3ec51c6cd1297071 not found: ID does not exist" containerID="583a3f000d9548e1a899cd2fe05f1a50ff08199784dd69bb3ec51c6cd1297071" Jul 02 22:10:23.284219 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:23.284183 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"583a3f000d9548e1a899cd2fe05f1a50ff08199784dd69bb3ec51c6cd1297071"} err="failed to get container status \"583a3f000d9548e1a899cd2fe05f1a50ff08199784dd69bb3ec51c6cd1297071\": rpc error: code = NotFound desc = could not find container \"583a3f000d9548e1a899cd2fe05f1a50ff08199784dd69bb3ec51c6cd1297071\": container with ID starting with 583a3f000d9548e1a899cd2fe05f1a50ff08199784dd69bb3ec51c6cd1297071 not found: ID does not exist" Jul 02 22:10:23.284219 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:23.284200 2556 scope.go:117] "RemoveContainer" containerID="97033d1d9e219e7bb60f7d111c8fcedcf4edc9f754c25f3c6cb9df9c38fa0ecc" Jul 02 22:10:23.284419 ip-10-0-140-146 kubenswrapper[2556]: E0702 22:10:23.284401 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97033d1d9e219e7bb60f7d111c8fcedcf4edc9f754c25f3c6cb9df9c38fa0ecc\": container with ID starting with 97033d1d9e219e7bb60f7d111c8fcedcf4edc9f754c25f3c6cb9df9c38fa0ecc not found: ID does not exist" containerID="97033d1d9e219e7bb60f7d111c8fcedcf4edc9f754c25f3c6cb9df9c38fa0ecc" Jul 02 22:10:23.284473 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:23.284427 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97033d1d9e219e7bb60f7d111c8fcedcf4edc9f754c25f3c6cb9df9c38fa0ecc"} err="failed to get container status \"97033d1d9e219e7bb60f7d111c8fcedcf4edc9f754c25f3c6cb9df9c38fa0ecc\": rpc error: code = NotFound desc = could not find container \"97033d1d9e219e7bb60f7d111c8fcedcf4edc9f754c25f3c6cb9df9c38fa0ecc\": container with ID starting with 97033d1d9e219e7bb60f7d111c8fcedcf4edc9f754c25f3c6cb9df9c38fa0ecc not found: ID does not exist" Jul 02 22:10:23.291542 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:23.291523 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl"] Jul 02 22:10:23.294102 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:23.294081 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-bd7159-openshift-default-76b4ccf66c-x24xl"] Jul 02 22:10:23.815187 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:23.815140 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a34d0412-2a2f-46df-97f8-46785985abbc" path="/var/lib/kubelet/pods/a34d0412-2a2f-46df-97f8-46785985abbc/volumes" Jul 02 22:10:24.022996 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:24.022960 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:24.121696 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:24.121600 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:24.301610 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:24.301579 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:24.530262 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:24.530230 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:25.007012 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.006988 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:25.343435 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.343350 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn"] Jul 02 22:10:25.343690 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.343672 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb31cfc4-33c5-412f-a457-ad10c9a587f8" containerName="maas-api" Jul 02 22:10:25.343765 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.343694 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb31cfc4-33c5-412f-a457-ad10c9a587f8" containerName="maas-api" Jul 02 22:10:25.343765 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.343708 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10834012-b7ac-43d8-879e-110239862715" containerName="maas-api" Jul 02 22:10:25.343765 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.343716 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="10834012-b7ac-43d8-879e-110239862715" containerName="maas-api" Jul 02 22:10:25.343765 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.343735 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a34d0412-2a2f-46df-97f8-46785985abbc" containerName="istio-proxy" Jul 02 22:10:25.343765 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.343744 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="a34d0412-2a2f-46df-97f8-46785985abbc" containerName="istio-proxy" Jul 02 22:10:25.343765 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.343755 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a34d0412-2a2f-46df-97f8-46785985abbc" containerName="istio-proxy" Jul 02 22:10:25.343765 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.343762 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="a34d0412-2a2f-46df-97f8-46785985abbc" containerName="istio-proxy" Jul 02 22:10:25.344134 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.343775 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c73068b3-2dd8-4c8c-bc74-5860f03aea13" containerName="istio-proxy" Jul 02 22:10:25.344134 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.343784 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="c73068b3-2dd8-4c8c-bc74-5860f03aea13" containerName="istio-proxy" Jul 02 22:10:25.344134 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.343861 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="a34d0412-2a2f-46df-97f8-46785985abbc" containerName="istio-proxy" Jul 02 22:10:25.344134 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.343889 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="10834012-b7ac-43d8-879e-110239862715" containerName="maas-api" Jul 02 22:10:25.344134 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.343901 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb31cfc4-33c5-412f-a457-ad10c9a587f8" containerName="maas-api" Jul 02 22:10:25.344134 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.343909 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="a34d0412-2a2f-46df-97f8-46785985abbc" containerName="istio-proxy" Jul 02 22:10:25.344134 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.343917 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="c73068b3-2dd8-4c8c-bc74-5860f03aea13" containerName="istio-proxy" Jul 02 22:10:25.347373 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.347335 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:10:25.353755 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.353738 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-edb549-openshift-default-dockercfg-74hrs\"" Jul 02 22:10:25.365684 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.365641 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:25.366665 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.366645 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn"] Jul 02 22:10:25.423476 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.423431 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fabfd48f-7fa3-47ba-ad2a-3a025256085e-istio-podinfo\") pod \"e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn\" (UID: \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\") " pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:10:25.423629 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.423481 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fabfd48f-7fa3-47ba-ad2a-3a025256085e-istiod-ca-cert\") pod \"e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn\" (UID: \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\") " pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:10:25.423629 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.423545 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fabfd48f-7fa3-47ba-ad2a-3a025256085e-istio-envoy\") pod \"e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn\" (UID: \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\") " pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:10:25.423629 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.423566 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2j8d\" (UniqueName: \"kubernetes.io/projected/fabfd48f-7fa3-47ba-ad2a-3a025256085e-kube-api-access-w2j8d\") pod \"e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn\" (UID: \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\") " pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:10:25.423629 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.423600 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fabfd48f-7fa3-47ba-ad2a-3a025256085e-credential-socket\") pod \"e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn\" (UID: \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\") " pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:10:25.423629 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.423625 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fabfd48f-7fa3-47ba-ad2a-3a025256085e-workload-socket\") pod \"e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn\" (UID: \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\") " pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:10:25.423786 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.423641 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fabfd48f-7fa3-47ba-ad2a-3a025256085e-istio-data\") pod \"e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn\" (UID: \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\") " pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:10:25.423786 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.423671 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fabfd48f-7fa3-47ba-ad2a-3a025256085e-istio-token\") pod \"e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn\" (UID: \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\") " pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:10:25.423786 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.423700 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fabfd48f-7fa3-47ba-ad2a-3a025256085e-workload-certs\") pod \"e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn\" (UID: \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\") " pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:10:25.524840 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.524808 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fabfd48f-7fa3-47ba-ad2a-3a025256085e-istiod-ca-cert\") pod \"e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn\" (UID: \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\") " pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:10:25.525022 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.524849 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fabfd48f-7fa3-47ba-ad2a-3a025256085e-istio-envoy\") pod \"e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn\" (UID: \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\") " pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:10:25.525022 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.524896 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2j8d\" (UniqueName: \"kubernetes.io/projected/fabfd48f-7fa3-47ba-ad2a-3a025256085e-kube-api-access-w2j8d\") pod \"e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn\" (UID: \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\") " pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:10:25.525022 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.524938 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fabfd48f-7fa3-47ba-ad2a-3a025256085e-credential-socket\") pod \"e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn\" (UID: \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\") " pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:10:25.525022 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.524975 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fabfd48f-7fa3-47ba-ad2a-3a025256085e-workload-socket\") pod \"e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn\" (UID: \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\") " pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:10:25.525022 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.524996 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fabfd48f-7fa3-47ba-ad2a-3a025256085e-istio-data\") pod \"e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn\" (UID: \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\") " pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:10:25.525022 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.525017 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fabfd48f-7fa3-47ba-ad2a-3a025256085e-istio-token\") pod \"e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn\" (UID: \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\") " pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:10:25.525333 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.525045 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fabfd48f-7fa3-47ba-ad2a-3a025256085e-workload-certs\") pod \"e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn\" (UID: \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\") " pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:10:25.525333 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.525087 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fabfd48f-7fa3-47ba-ad2a-3a025256085e-istio-podinfo\") pod \"e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn\" (UID: \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\") " pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:10:25.525444 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.525351 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fabfd48f-7fa3-47ba-ad2a-3a025256085e-credential-socket\") pod \"e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn\" (UID: \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\") " pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:10:25.525444 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.525429 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fabfd48f-7fa3-47ba-ad2a-3a025256085e-workload-socket\") pod \"e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn\" (UID: \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\") " pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:10:25.525562 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.525443 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fabfd48f-7fa3-47ba-ad2a-3a025256085e-istio-data\") pod \"e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn\" (UID: \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\") " pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:10:25.525644 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.525624 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fabfd48f-7fa3-47ba-ad2a-3a025256085e-workload-certs\") pod \"e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn\" (UID: \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\") " pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:10:25.525700 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.525647 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fabfd48f-7fa3-47ba-ad2a-3a025256085e-istiod-ca-cert\") pod \"e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn\" (UID: \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\") " pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:10:25.527324 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.527296 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fabfd48f-7fa3-47ba-ad2a-3a025256085e-istio-envoy\") pod \"e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn\" (UID: \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\") " pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:10:25.527620 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.527597 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fabfd48f-7fa3-47ba-ad2a-3a025256085e-istio-podinfo\") pod \"e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn\" (UID: \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\") " pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:10:25.535140 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.535117 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fabfd48f-7fa3-47ba-ad2a-3a025256085e-istio-token\") pod \"e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn\" (UID: \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\") " pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:10:25.535404 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.535387 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2j8d\" (UniqueName: \"kubernetes.io/projected/fabfd48f-7fa3-47ba-ad2a-3a025256085e-kube-api-access-w2j8d\") pod \"e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn\" (UID: \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\") " pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:10:25.539224 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.539175 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:25.657083 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.657015 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:10:25.791741 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.791689 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn"] Jul 02 22:10:25.794162 ip-10-0-140-146 kubenswrapper[2556]: W0702 22:10:25.794139 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfabfd48f_7fa3_47ba_ad2a_3a025256085e.slice/crio-ac127d8acbdb6cabe21283264170c1e5e2087773968f63c8a81fde952ae08407 WatchSource:0}: Error finding container ac127d8acbdb6cabe21283264170c1e5e2087773968f63c8a81fde952ae08407: Status 404 returned error can't find the container with id ac127d8acbdb6cabe21283264170c1e5e2087773968f63c8a81fde952ae08407 Jul 02 22:10:25.796170 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.796137 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 22:10:25.796258 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.796210 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 22:10:25.796258 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:25.796237 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 22:10:26.282515 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:26.282479 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" event={"ID":"fabfd48f-7fa3-47ba-ad2a-3a025256085e","Type":"ContainerStarted","Data":"a33559ae24027a0dcf97be1edebd6debf868e348af4f95d4b13ee211f7499642"} Jul 02 22:10:26.282961 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:26.282557 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" event={"ID":"fabfd48f-7fa3-47ba-ad2a-3a025256085e","Type":"ContainerStarted","Data":"ac127d8acbdb6cabe21283264170c1e5e2087773968f63c8a81fde952ae08407"} Jul 02 22:10:26.311091 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:26.311047 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" podStartSLOduration=1.311033249 podStartE2EDuration="1.311033249s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:10:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:10:26.309014407 +0000 UTC m=+2063.028363947" watchObservedRunningTime="2026-07-02 22:10:26.311033249 +0000 UTC m=+2063.030382792" Jul 02 22:10:26.658051 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:26.657970 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:10:26.662589 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:26.662566 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:10:27.286884 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:27.286837 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:10:27.287854 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:27.287835 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:10:28.398919 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:28.398858 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:28.460336 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:28.460303 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss"] Jul 02 22:10:28.462677 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:28.462659 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss" Jul 02 22:10:28.465307 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:28.465280 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-edb549-serving-cert\"" Jul 02 22:10:28.465447 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:28.465418 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-66gqh\"" Jul 02 22:10:28.472988 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:28.472968 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss"] Jul 02 22:10:28.495281 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:28.495254 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:28.547185 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:28.547156 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31453f0e-9154-4f0c-93df-8751ddd0e339-maas-api-tls\") pod \"maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss\" (UID: \"31453f0e-9154-4f0c-93df-8751ddd0e339\") " pod="opendatahub/maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss" Jul 02 22:10:28.547332 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:28.547200 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8t4p\" (UniqueName: \"kubernetes.io/projected/31453f0e-9154-4f0c-93df-8751ddd0e339-kube-api-access-k8t4p\") pod \"maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss\" (UID: \"31453f0e-9154-4f0c-93df-8751ddd0e339\") " pod="opendatahub/maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss" Jul 02 22:10:28.547332 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:28.547264 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31453f0e-9154-4f0c-93df-8751ddd0e339-openshift-service-ca\") pod \"maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss\" (UID: \"31453f0e-9154-4f0c-93df-8751ddd0e339\") " pod="opendatahub/maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss" Jul 02 22:10:28.595154 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:28.595120 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:28.648431 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:28.648403 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8t4p\" (UniqueName: \"kubernetes.io/projected/31453f0e-9154-4f0c-93df-8751ddd0e339-kube-api-access-k8t4p\") pod \"maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss\" (UID: \"31453f0e-9154-4f0c-93df-8751ddd0e339\") " pod="opendatahub/maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss" Jul 02 22:10:28.648548 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:28.648446 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31453f0e-9154-4f0c-93df-8751ddd0e339-openshift-service-ca\") pod \"maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss\" (UID: \"31453f0e-9154-4f0c-93df-8751ddd0e339\") " pod="opendatahub/maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss" Jul 02 22:10:28.648548 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:28.648497 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31453f0e-9154-4f0c-93df-8751ddd0e339-maas-api-tls\") pod \"maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss\" (UID: \"31453f0e-9154-4f0c-93df-8751ddd0e339\") " pod="opendatahub/maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss" Jul 02 22:10:28.648678 ip-10-0-140-146 kubenswrapper[2556]: E0702 22:10:28.648587 2556 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-a-edb549-serving-cert: secret "maas-api-e2e-shared-a-edb549-serving-cert" not found Jul 02 22:10:28.648678 ip-10-0-140-146 kubenswrapper[2556]: E0702 22:10:28.648641 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31453f0e-9154-4f0c-93df-8751ddd0e339-maas-api-tls podName:31453f0e-9154-4f0c-93df-8751ddd0e339 nodeName:}" failed. No retries permitted until 2026-07-02 22:10:29.148625549 +0000 UTC m=+2065.867975067 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/31453f0e-9154-4f0c-93df-8751ddd0e339-maas-api-tls") pod "maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss" (UID: "31453f0e-9154-4f0c-93df-8751ddd0e339") : secret "maas-api-e2e-shared-a-edb549-serving-cert" not found Jul 02 22:10:28.649224 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:28.649166 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31453f0e-9154-4f0c-93df-8751ddd0e339-openshift-service-ca\") pod \"maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss\" (UID: \"31453f0e-9154-4f0c-93df-8751ddd0e339\") " pod="opendatahub/maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss" Jul 02 22:10:28.657614 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:28.657597 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8t4p\" (UniqueName: \"kubernetes.io/projected/31453f0e-9154-4f0c-93df-8751ddd0e339-kube-api-access-k8t4p\") pod \"maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss\" (UID: \"31453f0e-9154-4f0c-93df-8751ddd0e339\") " pod="opendatahub/maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss" Jul 02 22:10:28.692993 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:28.692931 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:28.743687 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:28.743662 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:10:28.827244 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:28.827189 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:28.980388 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:28.980278 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:29.088978 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:29.088914 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:29.152715 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:29.152686 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31453f0e-9154-4f0c-93df-8751ddd0e339-maas-api-tls\") pod \"maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss\" (UID: \"31453f0e-9154-4f0c-93df-8751ddd0e339\") " pod="opendatahub/maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss" Jul 02 22:10:29.155236 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:29.155212 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31453f0e-9154-4f0c-93df-8751ddd0e339-maas-api-tls\") pod \"maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss\" (UID: \"31453f0e-9154-4f0c-93df-8751ddd0e339\") " pod="opendatahub/maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss" Jul 02 22:10:29.374045 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:29.374015 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss" Jul 02 22:10:29.504782 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:29.504668 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss"] Jul 02 22:10:29.507263 ip-10-0-140-146 kubenswrapper[2556]: W0702 22:10:29.507234 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31453f0e_9154_4f0c_93df_8751ddd0e339.slice/crio-376b3f8417dee43c31aac5f478531a3b0bf4320604ed99bcb30ff5c8352f81b5 WatchSource:0}: Error finding container 376b3f8417dee43c31aac5f478531a3b0bf4320604ed99bcb30ff5c8352f81b5: Status 404 returned error can't find the container with id 376b3f8417dee43c31aac5f478531a3b0bf4320604ed99bcb30ff5c8352f81b5 Jul 02 22:10:30.300936 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:30.300904 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss" event={"ID":"31453f0e-9154-4f0c-93df-8751ddd0e339","Type":"ContainerStarted","Data":"64278389e2889c914afe03f7d8b00f6fb29e8822804caac2b706071f4429f40a"} Jul 02 22:10:30.301076 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:30.300952 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss" event={"ID":"31453f0e-9154-4f0c-93df-8751ddd0e339","Type":"ContainerStarted","Data":"376b3f8417dee43c31aac5f478531a3b0bf4320604ed99bcb30ff5c8352f81b5"} Jul 02 22:10:30.301076 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:30.300994 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss" Jul 02 22:10:30.327910 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:30.327854 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss" podStartSLOduration=1.9591751 podStartE2EDuration="2.327832323s" totalImagesPullingTime="368.657223ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:10:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:10:30.326105658 +0000 UTC m=+2067.045455198" watchObservedRunningTime="2026-07-02 22:10:30.327832323 +0000 UTC m=+2067.047181864" Jul 02 22:10:32.881709 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:32.881646 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:33.220933 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:33.220828 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:33.607093 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:33.607054 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:33.915424 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:33.915342 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:36.316254 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:36.316227 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss" Jul 02 22:10:36.790342 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:36.790295 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:36.863285 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:36.863248 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs"] Jul 02 22:10:36.866134 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:36.866109 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs" Jul 02 22:10:36.869307 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:36.869286 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-f2a709-serving-cert\"" Jul 02 22:10:36.879204 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:36.879181 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs"] Jul 02 22:10:36.905736 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:36.905705 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:36.911121 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:36.911072 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe91d0fe-132f-4a89-ab6c-380ca10ac4c9-maas-api-tls\") pod \"maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs\" (UID: \"fe91d0fe-132f-4a89-ab6c-380ca10ac4c9\") " pod="opendatahub/maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs" Jul 02 22:10:36.911259 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:36.911206 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe91d0fe-132f-4a89-ab6c-380ca10ac4c9-openshift-service-ca\") pod \"maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs\" (UID: \"fe91d0fe-132f-4a89-ab6c-380ca10ac4c9\") " pod="opendatahub/maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs" Jul 02 22:10:36.911259 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:36.911233 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsk5k\" (UniqueName: \"kubernetes.io/projected/fe91d0fe-132f-4a89-ab6c-380ca10ac4c9-kube-api-access-dsk5k\") pod \"maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs\" (UID: \"fe91d0fe-132f-4a89-ab6c-380ca10ac4c9\") " pod="opendatahub/maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs" Jul 02 22:10:36.983195 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:36.983157 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:37.011819 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:37.011781 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe91d0fe-132f-4a89-ab6c-380ca10ac4c9-openshift-service-ca\") pod \"maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs\" (UID: \"fe91d0fe-132f-4a89-ab6c-380ca10ac4c9\") " pod="opendatahub/maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs" Jul 02 22:10:37.011819 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:37.011823 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsk5k\" (UniqueName: \"kubernetes.io/projected/fe91d0fe-132f-4a89-ab6c-380ca10ac4c9-kube-api-access-dsk5k\") pod \"maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs\" (UID: \"fe91d0fe-132f-4a89-ab6c-380ca10ac4c9\") " pod="opendatahub/maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs" Jul 02 22:10:37.012123 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:37.011858 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe91d0fe-132f-4a89-ab6c-380ca10ac4c9-maas-api-tls\") pod \"maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs\" (UID: \"fe91d0fe-132f-4a89-ab6c-380ca10ac4c9\") " pod="opendatahub/maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs" Jul 02 22:10:37.012123 ip-10-0-140-146 kubenswrapper[2556]: E0702 22:10:37.012087 2556 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-b-f2a709-serving-cert: secret "maas-api-e2e-shared-b-f2a709-serving-cert" not found Jul 02 22:10:37.012250 ip-10-0-140-146 kubenswrapper[2556]: E0702 22:10:37.012185 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe91d0fe-132f-4a89-ab6c-380ca10ac4c9-maas-api-tls podName:fe91d0fe-132f-4a89-ab6c-380ca10ac4c9 nodeName:}" failed. No retries permitted until 2026-07-02 22:10:37.512162553 +0000 UTC m=+2074.231512079 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fe91d0fe-132f-4a89-ab6c-380ca10ac4c9-maas-api-tls") pod "maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs" (UID: "fe91d0fe-132f-4a89-ab6c-380ca10ac4c9") : secret "maas-api-e2e-shared-b-f2a709-serving-cert" not found Jul 02 22:10:37.012556 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:37.012536 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe91d0fe-132f-4a89-ab6c-380ca10ac4c9-openshift-service-ca\") pod \"maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs\" (UID: \"fe91d0fe-132f-4a89-ab6c-380ca10ac4c9\") " pod="opendatahub/maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs" Jul 02 22:10:37.023613 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:37.023583 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsk5k\" (UniqueName: \"kubernetes.io/projected/fe91d0fe-132f-4a89-ab6c-380ca10ac4c9-kube-api-access-dsk5k\") pod \"maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs\" (UID: \"fe91d0fe-132f-4a89-ab6c-380ca10ac4c9\") " pod="opendatahub/maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs" Jul 02 22:10:37.346401 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:37.346371 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:37.447935 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:37.447901 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:37.515397 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:37.515361 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe91d0fe-132f-4a89-ab6c-380ca10ac4c9-maas-api-tls\") pod \"maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs\" (UID: \"fe91d0fe-132f-4a89-ab6c-380ca10ac4c9\") " pod="opendatahub/maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs" Jul 02 22:10:37.515568 ip-10-0-140-146 kubenswrapper[2556]: E0702 22:10:37.515517 2556 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-b-f2a709-serving-cert: secret "maas-api-e2e-shared-b-f2a709-serving-cert" not found Jul 02 22:10:37.515611 ip-10-0-140-146 kubenswrapper[2556]: E0702 22:10:37.515598 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe91d0fe-132f-4a89-ab6c-380ca10ac4c9-maas-api-tls podName:fe91d0fe-132f-4a89-ab6c-380ca10ac4c9 nodeName:}" failed. No retries permitted until 2026-07-02 22:10:38.515582121 +0000 UTC m=+2075.234931642 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fe91d0fe-132f-4a89-ab6c-380ca10ac4c9-maas-api-tls") pod "maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs" (UID: "fe91d0fe-132f-4a89-ab6c-380ca10ac4c9") : secret "maas-api-e2e-shared-b-f2a709-serving-cert" not found Jul 02 22:10:37.690320 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:37.690188 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:38.522625 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:38.522594 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe91d0fe-132f-4a89-ab6c-380ca10ac4c9-maas-api-tls\") pod \"maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs\" (UID: \"fe91d0fe-132f-4a89-ab6c-380ca10ac4c9\") " pod="opendatahub/maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs" Jul 02 22:10:38.525124 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:38.525104 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe91d0fe-132f-4a89-ab6c-380ca10ac4c9-maas-api-tls\") pod \"maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs\" (UID: \"fe91d0fe-132f-4a89-ab6c-380ca10ac4c9\") " pod="opendatahub/maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs" Jul 02 22:10:38.679279 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:38.679242 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs" Jul 02 22:10:38.844601 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:38.844579 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs"] Jul 02 22:10:38.847002 ip-10-0-140-146 kubenswrapper[2556]: W0702 22:10:38.846979 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe91d0fe_132f_4a89_ab6c_380ca10ac4c9.slice/crio-84985af4e013c2b1a75b3c94e86e9dc035814c67a01b391bd5b9135721135c9a WatchSource:0}: Error finding container 84985af4e013c2b1a75b3c94e86e9dc035814c67a01b391bd5b9135721135c9a: Status 404 returned error can't find the container with id 84985af4e013c2b1a75b3c94e86e9dc035814c67a01b391bd5b9135721135c9a Jul 02 22:10:39.345204 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:39.345172 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs" event={"ID":"fe91d0fe-132f-4a89-ab6c-380ca10ac4c9","Type":"ContainerStarted","Data":"af9793bea442bb8002c6915febc4a97ef961446c4bf32ddc3faa42a87f34a3a2"} Jul 02 22:10:39.345386 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:39.345224 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs" event={"ID":"fe91d0fe-132f-4a89-ab6c-380ca10ac4c9","Type":"ContainerStarted","Data":"84985af4e013c2b1a75b3c94e86e9dc035814c67a01b391bd5b9135721135c9a"} Jul 02 22:10:39.345386 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:39.345273 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs" Jul 02 22:10:39.363229 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:39.363180 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs" podStartSLOduration=3.046029221 podStartE2EDuration="3.363165161s" totalImagesPullingTime="317.13594ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:10:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:10:39.362352325 +0000 UTC m=+2076.081701866" watchObservedRunningTime="2026-07-02 22:10:39.363165161 +0000 UTC m=+2076.082514731" Jul 02 22:10:41.302209 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:41.302176 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:42.738424 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:42.738386 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:10:43.329049 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:43.329000 2556 patch_prober.go:28] interesting pod/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 02 22:10:43.329245 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:43.329068 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" podUID="fabfd48f-7fa3-47ba-ad2a-3a025256085e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 02 22:10:44.378859 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:44.378828 2556 generic.go:356] "Generic (PLEG): container finished" podID="fabfd48f-7fa3-47ba-ad2a-3a025256085e" containerID="a33559ae24027a0dcf97be1edebd6debf868e348af4f95d4b13ee211f7499642" exitCode=0 Jul 02 22:10:44.379289 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:44.378904 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" event={"ID":"fabfd48f-7fa3-47ba-ad2a-3a025256085e","Type":"ContainerDied","Data":"a33559ae24027a0dcf97be1edebd6debf868e348af4f95d4b13ee211f7499642"} Jul 02 22:10:44.379289 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:44.379234 2556 scope.go:117] "RemoveContainer" containerID="a33559ae24027a0dcf97be1edebd6debf868e348af4f95d4b13ee211f7499642" Jul 02 22:10:44.380207 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:44.380177 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 22:10:44.380354 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:44.380258 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 22:10:44.380354 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:44.380306 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 02 22:10:45.354832 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:45.354804 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs" Jul 02 22:10:45.388742 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:45.388706 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" event={"ID":"fabfd48f-7fa3-47ba-ad2a-3a025256085e","Type":"ContainerStarted","Data":"524fc8372ce2fdd39b4ac296487be30e9af216c7a9eb24a6299132fe2895bfd8"} Jul 02 22:10:45.657559 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:45.657472 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:10:45.657705 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:45.657670 2556 patch_prober.go:28] interesting pod/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Jul 02 22:10:45.657746 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:45.657717 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" podUID="fabfd48f-7fa3-47ba-ad2a-3a025256085e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jul 02 22:10:46.658122 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:46.658086 2556 patch_prober.go:28] interesting pod/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Jul 02 22:10:46.658522 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:46.658150 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" podUID="fabfd48f-7fa3-47ba-ad2a-3a025256085e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jul 02 22:10:47.657552 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:47.657519 2556 patch_prober.go:28] interesting pod/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Jul 02 22:10:47.657721 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:47.657570 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" podUID="fabfd48f-7fa3-47ba-ad2a-3a025256085e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jul 02 22:10:48.657821 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:48.657785 2556 patch_prober.go:28] interesting pod/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Jul 02 22:10:48.658242 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:48.657840 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" podUID="fabfd48f-7fa3-47ba-ad2a-3a025256085e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jul 02 22:10:49.660537 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:49.660506 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:10:49.661045 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:49.660860 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:10:49.661562 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:49.661542 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:10:49.815325 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:49.815291 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:49.922604 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:49.922488 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:52.084622 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:52.084589 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:58.246833 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:58.246756 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:58.345000 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:58.344969 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:10:58.933358 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:10:58.933327 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:11:03.815565 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:11:03.815530 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8jx8_7a32947e-1a4e-4138-8367-6def69748311/ovn-acl-logging/0.log" Jul 02 22:11:03.818792 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:11:03.818772 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8jx8_7a32947e-1a4e-4138-8367-6def69748311/ovn-acl-logging/0.log" Jul 02 22:11:18.134997 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:11:18.134956 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:11:18.190787 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:11:18.190764 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:11:19.367465 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:11:19.367435 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:11:23.034925 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:11:23.034895 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:11:53.538582 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:11:53.538549 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:12:12.539953 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:12:12.539924 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:12:28.536334 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:12:28.536300 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:12:46.735923 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:12:46.735887 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:13:04.340663 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:13:04.340629 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:13:20.645126 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:13:20.645090 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:13:35.545075 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:13:35.544993 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:13:36.377283 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:13:36.377247 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:13:39.543376 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:13:39.543342 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:13:47.484570 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:13:47.484519 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:13:54.343310 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:13:54.343278 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:14:00.037955 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:14:00.037921 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:14:10.594062 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:14:10.594025 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:14:15.641475 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:14:15.641444 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:14:38.949337 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:14:38.949301 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:14:57.455653 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:14:57.455551 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:14:58.558845 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:14:58.558807 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:14:59.969980 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:14:59.969934 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:15:00.151916 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:00.151861 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29717175-qht5x"] Jul 02 22:15:00.154412 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:00.154393 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717175-qht5x" Jul 02 22:15:00.165006 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:00.164987 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-edb549-29717175-6s57p"] Jul 02 22:15:00.167103 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:00.167090 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-edb549-29717175-6s57p" Jul 02 22:15:00.181260 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:00.181233 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29717175-qht5x"] Jul 02 22:15:00.182710 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:00.182688 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzvph\" (UniqueName: \"kubernetes.io/projected/34ac64a9-57c0-456a-a441-bc8c2519f946-kube-api-access-rzvph\") pod \"maas-api-key-cleanup-e2e-shared-a-edb549-29717175-6s57p\" (UID: \"34ac64a9-57c0-456a-a441-bc8c2519f946\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-edb549-29717175-6s57p" Jul 02 22:15:00.182838 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:00.182765 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79c96\" (UniqueName: \"kubernetes.io/projected/987ccaac-30ab-4b3c-aebd-b7bd01850e97-kube-api-access-79c96\") pod \"maas-api-key-cleanup-29717175-qht5x\" (UID: \"987ccaac-30ab-4b3c-aebd-b7bd01850e97\") " pod="opendatahub/maas-api-key-cleanup-29717175-qht5x" Jul 02 22:15:00.184380 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:00.184356 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-edb549-29717175-6s57p"] Jul 02 22:15:00.283740 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:00.283702 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79c96\" (UniqueName: \"kubernetes.io/projected/987ccaac-30ab-4b3c-aebd-b7bd01850e97-kube-api-access-79c96\") pod \"maas-api-key-cleanup-29717175-qht5x\" (UID: \"987ccaac-30ab-4b3c-aebd-b7bd01850e97\") " pod="opendatahub/maas-api-key-cleanup-29717175-qht5x" Jul 02 22:15:00.283925 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:00.283796 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzvph\" (UniqueName: \"kubernetes.io/projected/34ac64a9-57c0-456a-a441-bc8c2519f946-kube-api-access-rzvph\") pod \"maas-api-key-cleanup-e2e-shared-a-edb549-29717175-6s57p\" (UID: \"34ac64a9-57c0-456a-a441-bc8c2519f946\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-edb549-29717175-6s57p" Jul 02 22:15:00.298823 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:00.298790 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzvph\" (UniqueName: \"kubernetes.io/projected/34ac64a9-57c0-456a-a441-bc8c2519f946-kube-api-access-rzvph\") pod \"maas-api-key-cleanup-e2e-shared-a-edb549-29717175-6s57p\" (UID: \"34ac64a9-57c0-456a-a441-bc8c2519f946\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-edb549-29717175-6s57p" Jul 02 22:15:00.299760 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:00.299739 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-79c96\" (UniqueName: \"kubernetes.io/projected/987ccaac-30ab-4b3c-aebd-b7bd01850e97-kube-api-access-79c96\") pod \"maas-api-key-cleanup-29717175-qht5x\" (UID: \"987ccaac-30ab-4b3c-aebd-b7bd01850e97\") " pod="opendatahub/maas-api-key-cleanup-29717175-qht5x" Jul 02 22:15:00.463722 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:00.463683 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717175-qht5x" Jul 02 22:15:00.476094 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:00.476052 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-edb549-29717175-6s57p" Jul 02 22:15:00.631578 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:00.631549 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29717175-qht5x"] Jul 02 22:15:00.634322 ip-10-0-140-146 kubenswrapper[2556]: W0702 22:15:00.634295 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod987ccaac_30ab_4b3c_aebd_b7bd01850e97.slice/crio-a6756bb23bb4ae0e9b5de96da05b3d8762fbb98419b7b0e0894c2dd9f5161a60 WatchSource:0}: Error finding container a6756bb23bb4ae0e9b5de96da05b3d8762fbb98419b7b0e0894c2dd9f5161a60: Status 404 returned error can't find the container with id a6756bb23bb4ae0e9b5de96da05b3d8762fbb98419b7b0e0894c2dd9f5161a60 Jul 02 22:15:00.650259 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:00.650233 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-edb549-29717175-6s57p"] Jul 02 22:15:00.651610 ip-10-0-140-146 kubenswrapper[2556]: W0702 22:15:00.651589 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34ac64a9_57c0_456a_a441_bc8c2519f946.slice/crio-af2c56d78a5e1e27bc34ea065422b761c3d8804578eecab7492a3f1d840ae7ee WatchSource:0}: Error finding container af2c56d78a5e1e27bc34ea065422b761c3d8804578eecab7492a3f1d840ae7ee: Status 404 returned error can't find the container with id af2c56d78a5e1e27bc34ea065422b761c3d8804578eecab7492a3f1d840ae7ee Jul 02 22:15:01.421787 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:01.421739 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-edb549-29717175-6s57p" event={"ID":"34ac64a9-57c0-456a-a441-bc8c2519f946","Type":"ContainerStarted","Data":"5725b8b7218011625ef6e846df798ec723a9aaae8ba9aa2a9f7ae4bf1ae1df0f"} Jul 02 22:15:01.422334 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:01.421804 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-edb549-29717175-6s57p" event={"ID":"34ac64a9-57c0-456a-a441-bc8c2519f946","Type":"ContainerStarted","Data":"af2c56d78a5e1e27bc34ea065422b761c3d8804578eecab7492a3f1d840ae7ee"} Jul 02 22:15:01.423233 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:01.423205 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717175-qht5x" event={"ID":"987ccaac-30ab-4b3c-aebd-b7bd01850e97","Type":"ContainerStarted","Data":"c16eed959ccf540319153f9d54945decdb2e7c8020f563053e6c24e07339de37"} Jul 02 22:15:01.423346 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:01.423240 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717175-qht5x" event={"ID":"987ccaac-30ab-4b3c-aebd-b7bd01850e97","Type":"ContainerStarted","Data":"a6756bb23bb4ae0e9b5de96da05b3d8762fbb98419b7b0e0894c2dd9f5161a60"} Jul 02 22:15:01.509394 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:01.509339 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-edb549-29717175-6s57p" podStartSLOduration=1.509303523 podStartE2EDuration="1.509303523s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:15:01.460065073 +0000 UTC m=+2338.179414615" watchObservedRunningTime="2026-07-02 22:15:01.509303523 +0000 UTC m=+2338.228653064" Jul 02 22:15:01.509734 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:01.509714 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29717175-qht5x" podStartSLOduration=1.50970969 podStartE2EDuration="1.50970969s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:15:01.508141722 +0000 UTC m=+2338.227491263" watchObservedRunningTime="2026-07-02 22:15:01.50970969 +0000 UTC m=+2338.229059230" Jul 02 22:15:02.307847 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:02.307818 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:15:02.428862 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:02.428819 2556 generic.go:356] "Generic (PLEG): container finished" podID="987ccaac-30ab-4b3c-aebd-b7bd01850e97" containerID="c16eed959ccf540319153f9d54945decdb2e7c8020f563053e6c24e07339de37" exitCode=0 Jul 02 22:15:02.429284 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:02.428912 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717175-qht5x" event={"ID":"987ccaac-30ab-4b3c-aebd-b7bd01850e97","Type":"ContainerDied","Data":"c16eed959ccf540319153f9d54945decdb2e7c8020f563053e6c24e07339de37"} Jul 02 22:15:02.430391 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:02.430369 2556 generic.go:356] "Generic (PLEG): container finished" podID="34ac64a9-57c0-456a-a441-bc8c2519f946" containerID="5725b8b7218011625ef6e846df798ec723a9aaae8ba9aa2a9f7ae4bf1ae1df0f" exitCode=0 Jul 02 22:15:02.430514 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:02.430410 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-edb549-29717175-6s57p" event={"ID":"34ac64a9-57c0-456a-a441-bc8c2519f946","Type":"ContainerDied","Data":"5725b8b7218011625ef6e846df798ec723a9aaae8ba9aa2a9f7ae4bf1ae1df0f"} Jul 02 22:15:03.562969 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:03.562947 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-edb549-29717175-6s57p" Jul 02 22:15:03.580887 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:03.580851 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717175-qht5x" Jul 02 22:15:03.612359 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:03.612327 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79c96\" (UniqueName: \"kubernetes.io/projected/987ccaac-30ab-4b3c-aebd-b7bd01850e97-kube-api-access-79c96\") pod \"987ccaac-30ab-4b3c-aebd-b7bd01850e97\" (UID: \"987ccaac-30ab-4b3c-aebd-b7bd01850e97\") " Jul 02 22:15:03.612492 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:03.612414 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rzvph\" (UniqueName: \"kubernetes.io/projected/34ac64a9-57c0-456a-a441-bc8c2519f946-kube-api-access-rzvph\") pod \"34ac64a9-57c0-456a-a441-bc8c2519f946\" (UID: \"34ac64a9-57c0-456a-a441-bc8c2519f946\") " Jul 02 22:15:03.614679 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:03.614647 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34ac64a9-57c0-456a-a441-bc8c2519f946-kube-api-access-rzvph" (OuterVolumeSpecName: "kube-api-access-rzvph") pod "34ac64a9-57c0-456a-a441-bc8c2519f946" (UID: "34ac64a9-57c0-456a-a441-bc8c2519f946"). InnerVolumeSpecName "kube-api-access-rzvph". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:15:03.614826 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:03.614802 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/987ccaac-30ab-4b3c-aebd-b7bd01850e97-kube-api-access-79c96" (OuterVolumeSpecName: "kube-api-access-79c96") pod "987ccaac-30ab-4b3c-aebd-b7bd01850e97" (UID: "987ccaac-30ab-4b3c-aebd-b7bd01850e97"). InnerVolumeSpecName "kube-api-access-79c96". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:15:03.713472 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:03.713409 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rzvph\" (UniqueName: \"kubernetes.io/projected/34ac64a9-57c0-456a-a441-bc8c2519f946-kube-api-access-rzvph\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:15:03.713472 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:03.713431 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79c96\" (UniqueName: \"kubernetes.io/projected/987ccaac-30ab-4b3c-aebd-b7bd01850e97-kube-api-access-79c96\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:15:04.439475 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:04.439442 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-edb549-29717175-6s57p" Jul 02 22:15:04.439475 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:04.439461 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-edb549-29717175-6s57p" event={"ID":"34ac64a9-57c0-456a-a441-bc8c2519f946","Type":"ContainerDied","Data":"af2c56d78a5e1e27bc34ea065422b761c3d8804578eecab7492a3f1d840ae7ee"} Jul 02 22:15:04.439716 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:04.439510 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="af2c56d78a5e1e27bc34ea065422b761c3d8804578eecab7492a3f1d840ae7ee" Jul 02 22:15:04.440719 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:04.440693 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717175-qht5x" event={"ID":"987ccaac-30ab-4b3c-aebd-b7bd01850e97","Type":"ContainerDied","Data":"a6756bb23bb4ae0e9b5de96da05b3d8762fbb98419b7b0e0894c2dd9f5161a60"} Jul 02 22:15:04.440719 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:04.440711 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717175-qht5x" Jul 02 22:15:04.440893 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:04.440727 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a6756bb23bb4ae0e9b5de96da05b3d8762fbb98419b7b0e0894c2dd9f5161a60" Jul 02 22:15:11.803788 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:11.803751 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:15:11.935506 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:11.935472 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:15:17.440758 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:17.440722 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:15:19.633073 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:19.633037 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:15:34.865459 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:34.865419 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:15:39.151338 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:39.151301 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:15:50.171345 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:50.171315 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:15:50.540024 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:50.539988 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:15:50.640238 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:50.640200 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:15:58.968329 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:58.968281 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:15:59.078347 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:15:59.078312 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:16:00.742296 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:00.742266 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:16:03.839790 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:03.839762 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8jx8_7a32947e-1a4e-4138-8367-6def69748311/ovn-acl-logging/0.log" Jul 02 22:16:03.842986 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:03.842967 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8jx8_7a32947e-1a4e-4138-8367-6def69748311/ovn-acl-logging/0.log" Jul 02 22:16:23.442096 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:23.442065 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:16:27.857468 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:27.857381 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:16:28.660946 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:28.660911 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:16:39.050732 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:39.050700 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:16:54.436316 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:54.436282 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:16:54.844961 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:54.844927 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:16:54.994180 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:54.994148 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:16:55.097252 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:55.097177 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:16:55.221608 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:55.221578 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss"] Jul 02 22:16:55.221821 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:55.221795 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss" podUID="31453f0e-9154-4f0c-93df-8751ddd0e339" containerName="maas-api" containerID="cri-o://64278389e2889c914afe03f7d8b00f6fb29e8822804caac2b706071f4429f40a" gracePeriod=30 Jul 02 22:16:55.257126 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:55.257103 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-edb549-29717175-6s57p"] Jul 02 22:16:55.259677 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:55.259656 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:16:55.263248 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:55.263230 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-edb549-29717175-6s57p"] Jul 02 22:16:55.461889 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:55.461854 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss" Jul 02 22:16:55.587988 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:55.587962 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31453f0e-9154-4f0c-93df-8751ddd0e339-maas-api-tls\") pod \"31453f0e-9154-4f0c-93df-8751ddd0e339\" (UID: \"31453f0e-9154-4f0c-93df-8751ddd0e339\") " Jul 02 22:16:55.588149 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:55.588016 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31453f0e-9154-4f0c-93df-8751ddd0e339-openshift-service-ca\") pod \"31453f0e-9154-4f0c-93df-8751ddd0e339\" (UID: \"31453f0e-9154-4f0c-93df-8751ddd0e339\") " Jul 02 22:16:55.588149 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:55.588040 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8t4p\" (UniqueName: \"kubernetes.io/projected/31453f0e-9154-4f0c-93df-8751ddd0e339-kube-api-access-k8t4p\") pod \"31453f0e-9154-4f0c-93df-8751ddd0e339\" (UID: \"31453f0e-9154-4f0c-93df-8751ddd0e339\") " Jul 02 22:16:55.588345 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:55.588317 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31453f0e-9154-4f0c-93df-8751ddd0e339-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "31453f0e-9154-4f0c-93df-8751ddd0e339" (UID: "31453f0e-9154-4f0c-93df-8751ddd0e339"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:16:55.590189 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:55.590169 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31453f0e-9154-4f0c-93df-8751ddd0e339-kube-api-access-k8t4p" (OuterVolumeSpecName: "kube-api-access-k8t4p") pod "31453f0e-9154-4f0c-93df-8751ddd0e339" (UID: "31453f0e-9154-4f0c-93df-8751ddd0e339"). InnerVolumeSpecName "kube-api-access-k8t4p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:16:55.590277 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:55.590260 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31453f0e-9154-4f0c-93df-8751ddd0e339-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "31453f0e-9154-4f0c-93df-8751ddd0e339" (UID: "31453f0e-9154-4f0c-93df-8751ddd0e339"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 22:16:55.689343 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:55.689283 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31453f0e-9154-4f0c-93df-8751ddd0e339-maas-api-tls\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:16:55.689343 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:55.689304 2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31453f0e-9154-4f0c-93df-8751ddd0e339-openshift-service-ca\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:16:55.689343 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:55.689315 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8t4p\" (UniqueName: \"kubernetes.io/projected/31453f0e-9154-4f0c-93df-8751ddd0e339-kube-api-access-k8t4p\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:16:55.815486 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:55.815458 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34ac64a9-57c0-456a-a441-bc8c2519f946" path="/var/lib/kubelet/pods/34ac64a9-57c0-456a-a441-bc8c2519f946/volumes" Jul 02 22:16:55.892096 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:55.892063 2556 generic.go:356] "Generic (PLEG): container finished" podID="31453f0e-9154-4f0c-93df-8751ddd0e339" containerID="64278389e2889c914afe03f7d8b00f6fb29e8822804caac2b706071f4429f40a" exitCode=0 Jul 02 22:16:55.892225 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:55.892111 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss" Jul 02 22:16:55.892225 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:55.892141 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss" event={"ID":"31453f0e-9154-4f0c-93df-8751ddd0e339","Type":"ContainerDied","Data":"64278389e2889c914afe03f7d8b00f6fb29e8822804caac2b706071f4429f40a"} Jul 02 22:16:55.892225 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:55.892200 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss" event={"ID":"31453f0e-9154-4f0c-93df-8751ddd0e339","Type":"ContainerDied","Data":"376b3f8417dee43c31aac5f478531a3b0bf4320604ed99bcb30ff5c8352f81b5"} Jul 02 22:16:55.892326 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:55.892219 2556 scope.go:117] "RemoveContainer" containerID="64278389e2889c914afe03f7d8b00f6fb29e8822804caac2b706071f4429f40a" Jul 02 22:16:55.900183 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:55.900165 2556 scope.go:117] "RemoveContainer" containerID="64278389e2889c914afe03f7d8b00f6fb29e8822804caac2b706071f4429f40a" Jul 02 22:16:55.900422 ip-10-0-140-146 kubenswrapper[2556]: E0702 22:16:55.900405 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64278389e2889c914afe03f7d8b00f6fb29e8822804caac2b706071f4429f40a\": container with ID starting with 64278389e2889c914afe03f7d8b00f6fb29e8822804caac2b706071f4429f40a not found: ID does not exist" containerID="64278389e2889c914afe03f7d8b00f6fb29e8822804caac2b706071f4429f40a" Jul 02 22:16:55.900488 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:55.900431 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64278389e2889c914afe03f7d8b00f6fb29e8822804caac2b706071f4429f40a"} err="failed to get container status \"64278389e2889c914afe03f7d8b00f6fb29e8822804caac2b706071f4429f40a\": rpc error: code = NotFound desc = could not find container \"64278389e2889c914afe03f7d8b00f6fb29e8822804caac2b706071f4429f40a\": container with ID starting with 64278389e2889c914afe03f7d8b00f6fb29e8822804caac2b706071f4429f40a not found: ID does not exist" Jul 02 22:16:55.913279 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:55.913260 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss"] Jul 02 22:16:55.916813 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:55.916793 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-edb549-8446dd4b86-wfsss"] Jul 02 22:16:57.815578 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:57.815546 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31453f0e-9154-4f0c-93df-8751ddd0e339" path="/var/lib/kubelet/pods/31453f0e-9154-4f0c-93df-8751ddd0e339/volumes" Jul 02 22:16:58.840528 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:16:58.840496 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:17:01.241508 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:01.241472 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:17:01.400466 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:01.400435 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:17:01.523305 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:01.523270 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn"] Jul 02 22:17:01.523497 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:01.523476 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" podUID="fabfd48f-7fa3-47ba-ad2a-3a025256085e" containerName="istio-proxy" containerID="cri-o://524fc8372ce2fdd39b4ac296487be30e9af216c7a9eb24a6299132fe2895bfd8" gracePeriod=30 Jul 02 22:17:01.536578 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:01.536550 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:17:01.794124 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:01.794043 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:17:02.005624 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:02.005593 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:17:02.378288 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:02.378256 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:17:02.568049 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:02.568017 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:17:02.668654 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:02.668552 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:17:02.779137 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:02.779101 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs"] Jul 02 22:17:02.779363 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:02.779337 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs" podUID="fe91d0fe-132f-4a89-ab6c-380ca10ac4c9" containerName="maas-api" containerID="cri-o://af9793bea442bb8002c6915febc4a97ef961446c4bf32ddc3faa42a87f34a3a2" gracePeriod=30 Jul 02 22:17:02.874907 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:02.874859 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:17:02.922622 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:02.922544 2556 generic.go:356] "Generic (PLEG): container finished" podID="fe91d0fe-132f-4a89-ab6c-380ca10ac4c9" containerID="af9793bea442bb8002c6915febc4a97ef961446c4bf32ddc3faa42a87f34a3a2" exitCode=0 Jul 02 22:17:02.922747 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:02.922609 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs" event={"ID":"fe91d0fe-132f-4a89-ab6c-380ca10ac4c9","Type":"ContainerDied","Data":"af9793bea442bb8002c6915febc4a97ef961446c4bf32ddc3faa42a87f34a3a2"} Jul 02 22:17:03.024689 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:03.024665 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs" Jul 02 22:17:03.153932 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:03.153902 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe91d0fe-132f-4a89-ab6c-380ca10ac4c9-openshift-service-ca\") pod \"fe91d0fe-132f-4a89-ab6c-380ca10ac4c9\" (UID: \"fe91d0fe-132f-4a89-ab6c-380ca10ac4c9\") " Jul 02 22:17:03.154093 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:03.153971 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe91d0fe-132f-4a89-ab6c-380ca10ac4c9-maas-api-tls\") pod \"fe91d0fe-132f-4a89-ab6c-380ca10ac4c9\" (UID: \"fe91d0fe-132f-4a89-ab6c-380ca10ac4c9\") " Jul 02 22:17:03.154093 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:03.154006 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dsk5k\" (UniqueName: \"kubernetes.io/projected/fe91d0fe-132f-4a89-ab6c-380ca10ac4c9-kube-api-access-dsk5k\") pod \"fe91d0fe-132f-4a89-ab6c-380ca10ac4c9\" (UID: \"fe91d0fe-132f-4a89-ab6c-380ca10ac4c9\") " Jul 02 22:17:03.154286 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:03.154263 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe91d0fe-132f-4a89-ab6c-380ca10ac4c9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fe91d0fe-132f-4a89-ab6c-380ca10ac4c9" (UID: "fe91d0fe-132f-4a89-ab6c-380ca10ac4c9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:17:03.156122 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:03.156091 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe91d0fe-132f-4a89-ab6c-380ca10ac4c9-kube-api-access-dsk5k" (OuterVolumeSpecName: "kube-api-access-dsk5k") pod "fe91d0fe-132f-4a89-ab6c-380ca10ac4c9" (UID: "fe91d0fe-132f-4a89-ab6c-380ca10ac4c9"). InnerVolumeSpecName "kube-api-access-dsk5k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:17:03.156233 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:03.156186 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe91d0fe-132f-4a89-ab6c-380ca10ac4c9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fe91d0fe-132f-4a89-ab6c-380ca10ac4c9" (UID: "fe91d0fe-132f-4a89-ab6c-380ca10ac4c9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 22:17:03.254625 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:03.254570 2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe91d0fe-132f-4a89-ab6c-380ca10ac4c9-openshift-service-ca\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:17:03.254625 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:03.254591 2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe91d0fe-132f-4a89-ab6c-380ca10ac4c9-maas-api-tls\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:17:03.254625 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:03.254600 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dsk5k\" (UniqueName: \"kubernetes.io/projected/fe91d0fe-132f-4a89-ab6c-380ca10ac4c9-kube-api-access-dsk5k\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:17:03.916034 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:03.916009 2556 scope.go:117] "RemoveContainer" containerID="af9793bea442bb8002c6915febc4a97ef961446c4bf32ddc3faa42a87f34a3a2" Jul 02 22:17:03.928248 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:03.928230 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs" Jul 02 22:17:03.928337 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:03.928224 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs" event={"ID":"fe91d0fe-132f-4a89-ab6c-380ca10ac4c9","Type":"ContainerDied","Data":"84985af4e013c2b1a75b3c94e86e9dc035814c67a01b391bd5b9135721135c9a"} Jul 02 22:17:03.949531 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:03.949506 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs"] Jul 02 22:17:03.955369 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:03.955348 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-f2a709-5b5cf67f7f-w5fvs"] Jul 02 22:17:04.663350 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:04.663309 2556 patch_prober.go:28] interesting pod/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 02 22:17:04.663515 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:04.663374 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" podUID="fabfd48f-7fa3-47ba-ad2a-3a025256085e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 22:17:05.819403 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:05.819374 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe91d0fe-132f-4a89-ab6c-380ca10ac4c9" path="/var/lib/kubelet/pods/fe91d0fe-132f-4a89-ab6c-380ca10ac4c9/volumes" Jul 02 22:17:06.768910 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.768888 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:17:06.882061 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.881981 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2j8d\" (UniqueName: \"kubernetes.io/projected/fabfd48f-7fa3-47ba-ad2a-3a025256085e-kube-api-access-w2j8d\") pod \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\" (UID: \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\") " Jul 02 22:17:06.882061 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.882020 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fabfd48f-7fa3-47ba-ad2a-3a025256085e-istio-token\") pod \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\" (UID: \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\") " Jul 02 22:17:06.882061 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.882060 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fabfd48f-7fa3-47ba-ad2a-3a025256085e-workload-socket\") pod \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\" (UID: \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\") " Jul 02 22:17:06.882622 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.882098 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fabfd48f-7fa3-47ba-ad2a-3a025256085e-istio-podinfo\") pod \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\" (UID: \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\") " Jul 02 22:17:06.882622 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.882127 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fabfd48f-7fa3-47ba-ad2a-3a025256085e-istiod-ca-cert\") pod \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\" (UID: \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\") " Jul 02 22:17:06.882622 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.882152 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fabfd48f-7fa3-47ba-ad2a-3a025256085e-istio-data\") pod \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\" (UID: \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\") " Jul 02 22:17:06.882622 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.882188 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fabfd48f-7fa3-47ba-ad2a-3a025256085e-workload-certs\") pod \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\" (UID: \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\") " Jul 02 22:17:06.882622 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.882227 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fabfd48f-7fa3-47ba-ad2a-3a025256085e-istio-envoy\") pod \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\" (UID: \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\") " Jul 02 22:17:06.882622 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.882251 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fabfd48f-7fa3-47ba-ad2a-3a025256085e-credential-socket\") pod \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\" (UID: \"fabfd48f-7fa3-47ba-ad2a-3a025256085e\") " Jul 02 22:17:06.882622 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.882395 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fabfd48f-7fa3-47ba-ad2a-3a025256085e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fabfd48f-7fa3-47ba-ad2a-3a025256085e" (UID: "fabfd48f-7fa3-47ba-ad2a-3a025256085e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:17:06.882622 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.882501 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fabfd48f-7fa3-47ba-ad2a-3a025256085e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fabfd48f-7fa3-47ba-ad2a-3a025256085e" (UID: "fabfd48f-7fa3-47ba-ad2a-3a025256085e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:17:06.882622 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.882513 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fabfd48f-7fa3-47ba-ad2a-3a025256085e-workload-socket\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:17:06.883277 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.882757 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fabfd48f-7fa3-47ba-ad2a-3a025256085e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fabfd48f-7fa3-47ba-ad2a-3a025256085e" (UID: "fabfd48f-7fa3-47ba-ad2a-3a025256085e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:17:06.883277 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.882775 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fabfd48f-7fa3-47ba-ad2a-3a025256085e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fabfd48f-7fa3-47ba-ad2a-3a025256085e" (UID: "fabfd48f-7fa3-47ba-ad2a-3a025256085e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:17:06.883277 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.883118 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fabfd48f-7fa3-47ba-ad2a-3a025256085e-istio-data" (OuterVolumeSpecName: "istio-data") pod "fabfd48f-7fa3-47ba-ad2a-3a025256085e" (UID: "fabfd48f-7fa3-47ba-ad2a-3a025256085e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:17:06.884385 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.884360 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fabfd48f-7fa3-47ba-ad2a-3a025256085e-istio-token" (OuterVolumeSpecName: "istio-token") pod "fabfd48f-7fa3-47ba-ad2a-3a025256085e" (UID: "fabfd48f-7fa3-47ba-ad2a-3a025256085e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:17:06.884519 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.884403 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fabfd48f-7fa3-47ba-ad2a-3a025256085e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fabfd48f-7fa3-47ba-ad2a-3a025256085e" (UID: "fabfd48f-7fa3-47ba-ad2a-3a025256085e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 22:17:06.884772 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.884754 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fabfd48f-7fa3-47ba-ad2a-3a025256085e-kube-api-access-w2j8d" (OuterVolumeSpecName: "kube-api-access-w2j8d") pod "fabfd48f-7fa3-47ba-ad2a-3a025256085e" (UID: "fabfd48f-7fa3-47ba-ad2a-3a025256085e"). InnerVolumeSpecName "kube-api-access-w2j8d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:17:06.884842 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.884773 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fabfd48f-7fa3-47ba-ad2a-3a025256085e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fabfd48f-7fa3-47ba-ad2a-3a025256085e" (UID: "fabfd48f-7fa3-47ba-ad2a-3a025256085e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:17:06.941233 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.941205 2556 generic.go:356] "Generic (PLEG): container finished" podID="fabfd48f-7fa3-47ba-ad2a-3a025256085e" containerID="524fc8372ce2fdd39b4ac296487be30e9af216c7a9eb24a6299132fe2895bfd8" exitCode=0 Jul 02 22:17:06.941375 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.941262 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" Jul 02 22:17:06.941375 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.941278 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" event={"ID":"fabfd48f-7fa3-47ba-ad2a-3a025256085e","Type":"ContainerDied","Data":"524fc8372ce2fdd39b4ac296487be30e9af216c7a9eb24a6299132fe2895bfd8"} Jul 02 22:17:06.941375 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.941344 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn" event={"ID":"fabfd48f-7fa3-47ba-ad2a-3a025256085e","Type":"ContainerDied","Data":"ac127d8acbdb6cabe21283264170c1e5e2087773968f63c8a81fde952ae08407"} Jul 02 22:17:06.941375 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.941365 2556 scope.go:117] "RemoveContainer" containerID="524fc8372ce2fdd39b4ac296487be30e9af216c7a9eb24a6299132fe2895bfd8" Jul 02 22:17:06.949611 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.949587 2556 scope.go:117] "RemoveContainer" containerID="a33559ae24027a0dcf97be1edebd6debf868e348af4f95d4b13ee211f7499642" Jul 02 22:17:06.956701 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.956685 2556 scope.go:117] "RemoveContainer" containerID="524fc8372ce2fdd39b4ac296487be30e9af216c7a9eb24a6299132fe2895bfd8" Jul 02 22:17:06.956994 ip-10-0-140-146 kubenswrapper[2556]: E0702 22:17:06.956975 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"524fc8372ce2fdd39b4ac296487be30e9af216c7a9eb24a6299132fe2895bfd8\": container with ID starting with 524fc8372ce2fdd39b4ac296487be30e9af216c7a9eb24a6299132fe2895bfd8 not found: ID does not exist" containerID="524fc8372ce2fdd39b4ac296487be30e9af216c7a9eb24a6299132fe2895bfd8" Jul 02 22:17:06.957047 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.957002 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"524fc8372ce2fdd39b4ac296487be30e9af216c7a9eb24a6299132fe2895bfd8"} err="failed to get container status \"524fc8372ce2fdd39b4ac296487be30e9af216c7a9eb24a6299132fe2895bfd8\": rpc error: code = NotFound desc = could not find container \"524fc8372ce2fdd39b4ac296487be30e9af216c7a9eb24a6299132fe2895bfd8\": container with ID starting with 524fc8372ce2fdd39b4ac296487be30e9af216c7a9eb24a6299132fe2895bfd8 not found: ID does not exist" Jul 02 22:17:06.957047 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.957018 2556 scope.go:117] "RemoveContainer" containerID="a33559ae24027a0dcf97be1edebd6debf868e348af4f95d4b13ee211f7499642" Jul 02 22:17:06.957256 ip-10-0-140-146 kubenswrapper[2556]: E0702 22:17:06.957239 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a33559ae24027a0dcf97be1edebd6debf868e348af4f95d4b13ee211f7499642\": container with ID starting with a33559ae24027a0dcf97be1edebd6debf868e348af4f95d4b13ee211f7499642 not found: ID does not exist" containerID="a33559ae24027a0dcf97be1edebd6debf868e348af4f95d4b13ee211f7499642" Jul 02 22:17:06.957309 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.957265 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a33559ae24027a0dcf97be1edebd6debf868e348af4f95d4b13ee211f7499642"} err="failed to get container status \"a33559ae24027a0dcf97be1edebd6debf868e348af4f95d4b13ee211f7499642\": rpc error: code = NotFound desc = could not find container \"a33559ae24027a0dcf97be1edebd6debf868e348af4f95d4b13ee211f7499642\": container with ID starting with a33559ae24027a0dcf97be1edebd6debf868e348af4f95d4b13ee211f7499642 not found: ID does not exist" Jul 02 22:17:06.962976 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.962863 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn"] Jul 02 22:17:06.967185 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.967166 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-edb549-openshift-default-79b5774bbc-ng8bn"] Jul 02 22:17:06.982912 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.982863 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fabfd48f-7fa3-47ba-ad2a-3a025256085e-istio-envoy\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:17:06.982912 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.982909 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fabfd48f-7fa3-47ba-ad2a-3a025256085e-credential-socket\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:17:06.983037 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.982920 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2j8d\" (UniqueName: \"kubernetes.io/projected/fabfd48f-7fa3-47ba-ad2a-3a025256085e-kube-api-access-w2j8d\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:17:06.983037 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.982929 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fabfd48f-7fa3-47ba-ad2a-3a025256085e-istio-token\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:17:06.983037 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.982937 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fabfd48f-7fa3-47ba-ad2a-3a025256085e-istio-podinfo\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:17:06.983037 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.982944 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fabfd48f-7fa3-47ba-ad2a-3a025256085e-istiod-ca-cert\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:17:06.983037 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.982952 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fabfd48f-7fa3-47ba-ad2a-3a025256085e-istio-data\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:17:06.983037 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:06.982959 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fabfd48f-7fa3-47ba-ad2a-3a025256085e-workload-certs\") on node \"ip-10-0-140-146.ec2.internal\" DevicePath \"\"" Jul 02 22:17:07.814996 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:07.814967 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fabfd48f-7fa3-47ba-ad2a-3a025256085e" path="/var/lib/kubelet/pods/fabfd48f-7fa3-47ba-ad2a-3a025256085e/volumes" Jul 02 22:17:08.794197 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:08.794166 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:17:08.889344 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:08.889315 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:17:09.120051 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:09.119959 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:17:09.342754 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:09.342719 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:17:09.588291 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:09.588259 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:17:09.668849 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:09.668813 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:17:11.612813 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:11.612781 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-k8cbc"] Jul 02 22:17:14.843835 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:14.843808 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29717160-lg8nj_80226df1-8f5a-42ce-8d75-280a5cb5fab8/cleanup/0.log" Jul 02 22:17:14.963785 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:14.963754 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29717175-qht5x_987ccaac-30ab-4b3c-aebd-b7bd01850e97/cleanup/0.log" Jul 02 22:17:15.417050 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:15.417024 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bcc4b86cc-27xdp_6360a44c-1e00-45a6-8ba0-42b74f41bba5/copy-manifests/0.log" Jul 02 22:17:15.430986 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:15.430962 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bcc4b86cc-27xdp_6360a44c-1e00-45a6-8ba0-42b74f41bba5/manager/0.log" Jul 02 22:17:17.935408 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:17.935375 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-k8cbc_9e8af168-3721-4834-adf0-e3e4b4922633/limitador/0.log" Jul 02 22:17:18.537343 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:18.537300 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-44zd7_5256ef8e-3e79-4873-8cde-41997843d404/discovery/0.log" Jul 02 22:17:19.976407 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:19.976374 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbg8hk8_1d5bfc38-0ccf-4559-aa72-ff33de61fef8/main/0.log" Jul 02 22:17:24.302128 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:24.302098 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:17:24.397022 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:24.396993 2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" limit=1024 Jul 02 22:17:27.322888 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:27.322839 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-bg9p7_27a3595e-6df7-4999-8257-5c111b074330/global-pull-secret-syncer/0.log" Jul 02 22:17:27.421377 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:27.421346 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-mtlf9_13dcc457-ac6d-416b-82e8-191fa49d0349/konnectivity-agent/0.log" Jul 02 22:17:27.563253 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:27.563225 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-146.ec2.internal_8861d880e93a504d75b18448f12e97fd/haproxy/0.log" Jul 02 22:17:33.196945 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:33.196916 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-k8cbc_9e8af168-3721-4834-adf0-e3e4b4922633/limitador/0.log" Jul 02 22:17:35.805462 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:35.805434 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gwshn_d4eab772-b0dd-4373-86f5-1ea1b897b14f/node-exporter/0.log" Jul 02 22:17:35.871625 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:35.871596 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gwshn_d4eab772-b0dd-4373-86f5-1ea1b897b14f/kube-rbac-proxy/0.log" Jul 02 22:17:35.917209 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:35.917181 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gwshn_d4eab772-b0dd-4373-86f5-1ea1b897b14f/init-textfile/0.log" Jul 02 22:17:37.080102 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.080068 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-h24vs/perf-node-gather-daemonset-4pjpx"] Jul 02 22:17:37.080498 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.080336 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="987ccaac-30ab-4b3c-aebd-b7bd01850e97" containerName="cleanup" Jul 02 22:17:37.080498 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.080346 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="987ccaac-30ab-4b3c-aebd-b7bd01850e97" containerName="cleanup" Jul 02 22:17:37.080498 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.080362 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fabfd48f-7fa3-47ba-ad2a-3a025256085e" containerName="istio-proxy" Jul 02 22:17:37.080498 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.080368 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="fabfd48f-7fa3-47ba-ad2a-3a025256085e" containerName="istio-proxy" Jul 02 22:17:37.080498 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.080375 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fabfd48f-7fa3-47ba-ad2a-3a025256085e" containerName="istio-proxy" Jul 02 22:17:37.080498 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.080380 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="fabfd48f-7fa3-47ba-ad2a-3a025256085e" containerName="istio-proxy" Jul 02 22:17:37.080498 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.080386 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34ac64a9-57c0-456a-a441-bc8c2519f946" containerName="cleanup" Jul 02 22:17:37.080498 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.080391 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="34ac64a9-57c0-456a-a441-bc8c2519f946" containerName="cleanup" Jul 02 22:17:37.080498 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.080399 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31453f0e-9154-4f0c-93df-8751ddd0e339" containerName="maas-api" Jul 02 22:17:37.080498 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.080405 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="31453f0e-9154-4f0c-93df-8751ddd0e339" containerName="maas-api" Jul 02 22:17:37.080498 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.080413 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe91d0fe-132f-4a89-ab6c-380ca10ac4c9" containerName="maas-api" Jul 02 22:17:37.080498 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.080418 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe91d0fe-132f-4a89-ab6c-380ca10ac4c9" containerName="maas-api" Jul 02 22:17:37.080498 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.080460 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="34ac64a9-57c0-456a-a441-bc8c2519f946" containerName="cleanup" Jul 02 22:17:37.080498 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.080467 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="31453f0e-9154-4f0c-93df-8751ddd0e339" containerName="maas-api" Jul 02 22:17:37.080498 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.080473 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe91d0fe-132f-4a89-ab6c-380ca10ac4c9" containerName="maas-api" Jul 02 22:17:37.080498 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.080478 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="fabfd48f-7fa3-47ba-ad2a-3a025256085e" containerName="istio-proxy" Jul 02 22:17:37.080498 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.080483 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="fabfd48f-7fa3-47ba-ad2a-3a025256085e" containerName="istio-proxy" Jul 02 22:17:37.080498 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.080488 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="987ccaac-30ab-4b3c-aebd-b7bd01850e97" containerName="cleanup" Jul 02 22:17:37.082553 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.082518 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-4pjpx" Jul 02 22:17:37.085333 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.085313 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-h24vs\"/\"kube-root-ca.crt\"" Jul 02 22:17:37.086474 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.086450 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-h24vs\"/\"openshift-service-ca.crt\"" Jul 02 22:17:37.086587 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.086450 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-h24vs\"/\"default-dockercfg-6wgd7\"" Jul 02 22:17:37.093146 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.093125 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-h24vs/perf-node-gather-daemonset-4pjpx"] Jul 02 22:17:37.187022 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.186990 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dcfd4532-aac3-4609-940b-5ad21ab31784-sys\") pod \"perf-node-gather-daemonset-4pjpx\" (UID: \"dcfd4532-aac3-4609-940b-5ad21ab31784\") " pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-4pjpx" Jul 02 22:17:37.187022 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.187036 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dcfd4532-aac3-4609-940b-5ad21ab31784-lib-modules\") pod \"perf-node-gather-daemonset-4pjpx\" (UID: \"dcfd4532-aac3-4609-940b-5ad21ab31784\") " pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-4pjpx" Jul 02 22:17:37.187255 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.187063 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ht6jq\" (UniqueName: \"kubernetes.io/projected/dcfd4532-aac3-4609-940b-5ad21ab31784-kube-api-access-ht6jq\") pod \"perf-node-gather-daemonset-4pjpx\" (UID: \"dcfd4532-aac3-4609-940b-5ad21ab31784\") " pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-4pjpx" Jul 02 22:17:37.187255 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.187132 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/dcfd4532-aac3-4609-940b-5ad21ab31784-podres\") pod \"perf-node-gather-daemonset-4pjpx\" (UID: \"dcfd4532-aac3-4609-940b-5ad21ab31784\") " pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-4pjpx" Jul 02 22:17:37.187255 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.187195 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/dcfd4532-aac3-4609-940b-5ad21ab31784-proc\") pod \"perf-node-gather-daemonset-4pjpx\" (UID: \"dcfd4532-aac3-4609-940b-5ad21ab31784\") " pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-4pjpx" Jul 02 22:17:37.287943 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.287909 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/dcfd4532-aac3-4609-940b-5ad21ab31784-podres\") pod \"perf-node-gather-daemonset-4pjpx\" (UID: \"dcfd4532-aac3-4609-940b-5ad21ab31784\") " pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-4pjpx" Jul 02 22:17:37.288122 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.287964 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/dcfd4532-aac3-4609-940b-5ad21ab31784-proc\") pod \"perf-node-gather-daemonset-4pjpx\" (UID: \"dcfd4532-aac3-4609-940b-5ad21ab31784\") " pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-4pjpx" Jul 02 22:17:37.288122 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.287981 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dcfd4532-aac3-4609-940b-5ad21ab31784-sys\") pod \"perf-node-gather-daemonset-4pjpx\" (UID: \"dcfd4532-aac3-4609-940b-5ad21ab31784\") " pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-4pjpx" Jul 02 22:17:37.288122 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.288012 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dcfd4532-aac3-4609-940b-5ad21ab31784-lib-modules\") pod \"perf-node-gather-daemonset-4pjpx\" (UID: \"dcfd4532-aac3-4609-940b-5ad21ab31784\") " pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-4pjpx" Jul 02 22:17:37.288122 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.288033 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ht6jq\" (UniqueName: \"kubernetes.io/projected/dcfd4532-aac3-4609-940b-5ad21ab31784-kube-api-access-ht6jq\") pod \"perf-node-gather-daemonset-4pjpx\" (UID: \"dcfd4532-aac3-4609-940b-5ad21ab31784\") " pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-4pjpx" Jul 02 22:17:37.288122 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.288064 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/dcfd4532-aac3-4609-940b-5ad21ab31784-proc\") pod \"perf-node-gather-daemonset-4pjpx\" (UID: \"dcfd4532-aac3-4609-940b-5ad21ab31784\") " pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-4pjpx" Jul 02 22:17:37.288122 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.288073 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/dcfd4532-aac3-4609-940b-5ad21ab31784-podres\") pod \"perf-node-gather-daemonset-4pjpx\" (UID: \"dcfd4532-aac3-4609-940b-5ad21ab31784\") " pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-4pjpx" Jul 02 22:17:37.288334 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.288127 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dcfd4532-aac3-4609-940b-5ad21ab31784-sys\") pod \"perf-node-gather-daemonset-4pjpx\" (UID: \"dcfd4532-aac3-4609-940b-5ad21ab31784\") " pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-4pjpx" Jul 02 22:17:37.288334 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.288177 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dcfd4532-aac3-4609-940b-5ad21ab31784-lib-modules\") pod \"perf-node-gather-daemonset-4pjpx\" (UID: \"dcfd4532-aac3-4609-940b-5ad21ab31784\") " pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-4pjpx" Jul 02 22:17:37.297390 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.297372 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ht6jq\" (UniqueName: \"kubernetes.io/projected/dcfd4532-aac3-4609-940b-5ad21ab31784-kube-api-access-ht6jq\") pod \"perf-node-gather-daemonset-4pjpx\" (UID: \"dcfd4532-aac3-4609-940b-5ad21ab31784\") " pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-4pjpx" Jul 02 22:17:37.392683 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.392620 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-4pjpx" Jul 02 22:17:37.518432 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:37.518407 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-h24vs/perf-node-gather-daemonset-4pjpx"] Jul 02 22:17:37.520925 ip-10-0-140-146 kubenswrapper[2556]: W0702 22:17:37.520903 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poddcfd4532_aac3_4609_940b_5ad21ab31784.slice/crio-5a3666f2c25fe4838ad2107792065c60e02052535ad35bdb7a9ede25001a63bb WatchSource:0}: Error finding container 5a3666f2c25fe4838ad2107792065c60e02052535ad35bdb7a9ede25001a63bb: Status 404 returned error can't find the container with id 5a3666f2c25fe4838ad2107792065c60e02052535ad35bdb7a9ede25001a63bb Jul 02 22:17:38.065793 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:38.065753 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-4pjpx" event={"ID":"dcfd4532-aac3-4609-940b-5ad21ab31784","Type":"ContainerStarted","Data":"bcb747a8c3a3e990edc948fbbbc86bb1016d595d76afe81ad2b3b99b2bb7600c"} Jul 02 22:17:38.066013 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:38.065812 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-4pjpx" event={"ID":"dcfd4532-aac3-4609-940b-5ad21ab31784","Type":"ContainerStarted","Data":"5a3666f2c25fe4838ad2107792065c60e02052535ad35bdb7a9ede25001a63bb"} Jul 02 22:17:38.066013 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:38.065892 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-4pjpx" Jul 02 22:17:38.085210 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:38.085152 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-4pjpx" podStartSLOduration=1.085138948 podStartE2EDuration="1.085138948s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 22:17:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:17:38.084092728 +0000 UTC m=+2494.803442268" watchObservedRunningTime="2026-07-02 22:17:38.085138948 +0000 UTC m=+2494.804488488" Jul 02 22:17:40.483791 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:40.483762 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-m7c4n_487a1265-7879-4ca9-a747-40bbdba201c4/dns/0.log" Jul 02 22:17:40.506127 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:40.506103 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-m7c4n_487a1265-7879-4ca9-a747-40bbdba201c4/kube-rbac-proxy/0.log" Jul 02 22:17:40.657715 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:40.657683 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jgmz2_222d3332-3472-479e-a8e2-8d8d9c702a9b/dns-node-resolver/0.log" Jul 02 22:17:41.273860 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:41.273836 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-82ctv_baf560bd-eaae-468e-bc1c-a007fe68bded/node-ca/0.log" Jul 02 22:17:42.411766 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:42.411734 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-44zd7_5256ef8e-3e79-4873-8cde-41997843d404/discovery/0.log" Jul 02 22:17:43.346279 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:43.346247 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qcb9v_5a8c260d-9be7-40ca-a29b-edb3cf420235/serve-healthcheck-canary/0.log" Jul 02 22:17:44.080421 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:44.080397 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-h24vs/perf-node-gather-daemonset-4pjpx" Jul 02 22:17:44.086365 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:44.086304 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lzqbp_245f2091-5cf4-4185-a0fb-629ec98a0be0/kube-rbac-proxy/0.log" Jul 02 22:17:44.109729 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:44.109705 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lzqbp_245f2091-5cf4-4185-a0fb-629ec98a0be0/exporter/0.log" Jul 02 22:17:44.133199 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:44.133178 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lzqbp_245f2091-5cf4-4185-a0fb-629ec98a0be0/extractor/0.log" Jul 02 22:17:46.796295 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:46.796259 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29717160-lg8nj_80226df1-8f5a-42ce-8d75-280a5cb5fab8/cleanup/0.log" Jul 02 22:17:46.829750 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:46.829723 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29717175-qht5x_987ccaac-30ab-4b3c-aebd-b7bd01850e97/cleanup/0.log" Jul 02 22:17:47.085622 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:47.085592 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bcc4b86cc-27xdp_6360a44c-1e00-45a6-8ba0-42b74f41bba5/manager/0.log" Jul 02 22:17:47.112386 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:47.112357 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bcc4b86cc-27xdp_6360a44c-1e00-45a6-8ba0-42b74f41bba5/copy-manifests/0.log" Jul 02 22:17:47.591175 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:47.591147 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-44zd7_5256ef8e-3e79-4873-8cde-41997843d404/discovery/0.log" Jul 02 22:17:48.773146 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:48.773122 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-8677d7b8dc-xtgj6_d79c0f71-febe-42f6-94eb-728225cd99fe/manager/0.log" Jul 02 22:17:49.120911 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:49.120816 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-44zd7_5256ef8e-3e79-4873-8cde-41997843d404/discovery/0.log" Jul 02 22:17:53.602585 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:53.602523 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-xb5mh_c96f6156-aad2-4db8-af99-ea5f38b27ed2/migrator/0.log" Jul 02 22:17:53.628770 ip-10-0-140-146 kubenswrapper[2556]: I0702 22:17:53.628744 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-xb5mh_c96f6156-aad2-4db8-af99-ea5f38b27ed2/graceful-termination/0.log"